-- Logs begin at Mon 2022-08-22 18:29:05 UTC, end at Mon 2022-08-22 20:15:59 UTC. -- Aug 22 19:04:21.571104 dv-cinder-835384-22-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 22 19:04:21.578904 dv-cinder-835384-22-iscsi devstack@keystone.service[81520]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 22 19:04:21.585235 dv-cinder-835384-22-iscsi devstack@keystone.service[81520]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Aug 22 19:04:21 2022] *** Aug 22 19:04:21.585235 dv-cinder-835384-22-iscsi devstack@keystone.service[81520]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 22 19:04:21.585235 dv-cinder-835384-22-iscsi devstack@keystone.service[81520]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 22 19:04:21.585235 dv-cinder-835384-22-iscsi devstack@keystone.service[81520]: nodename: dv-cinder-835384-22-iscsi Aug 22 19:04:21.585235 dv-cinder-835384-22-iscsi devstack@keystone.service[81520]: machine: x86_64 Aug 22 19:04:21.585235 dv-cinder-835384-22-iscsi devstack@keystone.service[81520]: clock source: unix Aug 22 19:04:21.585235 dv-cinder-835384-22-iscsi devstack@keystone.service[81520]: pcre jit disabled Aug 22 19:04:21.585235 dv-cinder-835384-22-iscsi devstack@keystone.service[81520]: detected number of CPU cores: 4 Aug 22 19:04:21.585235 dv-cinder-835384-22-iscsi devstack@keystone.service[81520]: current working directory: / Aug 22 19:04:21.585235 dv-cinder-835384-22-iscsi devstack@keystone.service[81520]: detected binary path: /usr/bin/uwsgi-core Aug 22 19:04:21.585235 dv-cinder-835384-22-iscsi devstack@keystone.service[81520]: your processes number limit is 47680 Aug 22 19:04:21.585235 dv-cinder-835384-22-iscsi devstack@keystone.service[81520]: your memory page size is 4096 bytes Aug 22 19:04:21.585235 dv-cinder-835384-22-iscsi devstack@keystone.service[81520]: detected max file descriptor number: 2048 Aug 22 19:04:21.585235 dv-cinder-835384-22-iscsi devstack@keystone.service[81520]: lock engine: pthread robust mutexes Aug 22 19:04:21.585235 dv-cinder-835384-22-iscsi devstack@keystone.service[81520]: thunder lock: enabled Aug 22 19:04:21.585235 dv-cinder-835384-22-iscsi devstack@keystone.service[81520]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 22 19:04:21.585235 dv-cinder-835384-22-iscsi devstack@keystone.service[81520]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 22 19:04:21.622028 dv-cinder-835384-22-iscsi devstack@keystone.service[81520]: Python main interpreter initialized at 0x55d7dda8e9c0 Aug 22 19:04:21.622028 dv-cinder-835384-22-iscsi devstack@keystone.service[81520]: python threads support enabled Aug 22 19:04:21.622028 dv-cinder-835384-22-iscsi devstack@keystone.service[81520]: your server socket listen backlog is limited to 100 connections Aug 22 19:04:21.622028 dv-cinder-835384-22-iscsi devstack@keystone.service[81520]: your mercy for graceful operations on workers is 90 seconds Aug 22 19:04:21.622363 dv-cinder-835384-22-iscsi devstack@keystone.service[81520]: mapped 403077 bytes (393 KB) for 2 cores Aug 22 19:04:21.622474 dv-cinder-835384-22-iscsi devstack@keystone.service[81520]: *** Operational MODE: preforking *** Aug 22 19:04:21.622746 dv-cinder-835384-22-iscsi devstack@keystone.service[81520]: *** uWSGI is running in multiple interpreter mode *** Aug 22 19:04:21.622746 dv-cinder-835384-22-iscsi devstack@keystone.service[81520]: spawned uWSGI master process (pid: 81520) Aug 22 19:04:21.622946 dv-cinder-835384-22-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 22 19:04:21.624057 dv-cinder-835384-22-iscsi devstack@keystone.service[81520]: spawned uWSGI worker 1 (pid: 81532, cores: 1) Aug 22 19:04:21.624057 dv-cinder-835384-22-iscsi devstack@keystone.service[81520]: spawned uWSGI worker 2 (pid: 81533, cores: 1) Aug 22 19:04:21.624057 dv-cinder-835384-22-iscsi devstack@keystone.service[81520]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 22 19:04:25.876593 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 22 19:04:25.876593 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: from cryptography.utils import int_from_bytes Aug 22 19:04:25.877385 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 22 19:04:25.877385 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: from cryptography.utils import int_from_bytes Aug 22 19:04:25.877533 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 22 19:04:25.877533 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: from cryptography.utils import int_from_bytes Aug 22 19:04:25.877533 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 22 19:04:25.877533 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: from cryptography.utils import int_from_bytes Aug 22 19:04:25.926944 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 22 19:04:25.927138 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 22 19:04:25.927326 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] command line args: [] {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 22 19:04:25.927463 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 22 19:04:25.927593 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 22 19:04:25.927701 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 22 19:04:25.927784 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ================================================================================ {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 22 19:04:25.927867 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] command line args: [] {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 22 19:04:25.928032 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] admin_token = **** {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 19:04:25.928112 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 22 19:04:25.928274 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] config_dir = [] {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 19:04:25.928360 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ================================================================================ {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 22 19:04:25.928457 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 19:04:25.928548 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] admin_token = **** {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 19:04:25.928621 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] config_source = [] {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 19:04:25.928761 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] config_dir = [] {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 19:04:25.928846 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] debug = True {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 19:04:25.928951 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 19:04:25.929030 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 19:04:25.929229 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] config_source = [] {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 19:04:25.929301 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] debug = True {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 19:04:25.929362 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 19:04:25.929491 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 19:04:25.929606 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 19:04:25.929718 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 19:04:25.929815 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 19:04:25.929969 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 19:04:25.930082 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 19:04:25.930188 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 19:04:25.930298 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] list_limit = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 19:04:25.930404 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] log_config_append = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 19:04:25.930518 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 19:04:25.930623 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 19:04:25.930721 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] list_limit = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 19:04:25.930831 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] log_dir = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 19:04:25.930940 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] log_config_append = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 19:04:25.931045 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] log_file = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 19:04:25.931144 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 19:04:25.931248 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 19:04:25.931329 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] log_dir = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 19:04:25.931329 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] log_file = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 19:04:25.931526 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 19:04:25.931600 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 19:04:25.931752 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 19:04:25.931836 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 19:04:25.931916 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 19:04:25.931916 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 19:04:25.932120 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 19:04:25.932200 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 19:04:25.932280 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 19:04:25.932408 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 19:04:25.932485 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 19:04:25.932485 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 19:04:25.932683 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 19:04:25.932763 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 19:04:25.932886 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 19:04:25.932974 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 19:04:25.933049 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 19:04:25.933177 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 19:04:25.933261 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 19:04:25.933396 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 19:04:25.933483 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 19:04:25.933630 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 19:04:25.933758 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 19:04:25.933847 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 19:04:25.933943 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 19:04:25.934097 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 19:04:25.934183 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 19:04:25.934342 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 19:04:25.934610 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.61/identity {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 19:04:25.934705 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 19:04:25.934780 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] publish_errors = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 19:04:25.935006 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 19:04:25.935092 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.61/identity {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 19:04:25.935234 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 19:04:25.935320 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] publish_errors = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 19:04:25.935399 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 19:04:25.935523 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 19:04:25.935658 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 19:04:25.935752 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 19:04:25.935895 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 19:04:25.935992 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 19:04:25.936126 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] standard_threads = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 19:04:25.936205 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 19:04:25.936352 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 19:04:25.936430 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 19:04:25.936586 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] standard_threads = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 19:04:25.936675 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 19:04:25.936766 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 19:04:25.936840 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 19:04:25.936944 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] use_journal = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 19:04:25.937080 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 19:04:25.937165 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] use_json = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 19:04:25.937258 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 19:04:25.937401 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] use_stderr = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 19:04:25.937478 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] use_syslog = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 19:04:25.937555 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] use_journal = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 19:04:25.937628 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] use_json = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 19:04:25.937704 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 19:04:25.937782 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] use_stderr = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 19:04:25.937865 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.937939 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] use_syslog = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 19:04:25.938105 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.938184 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 19:04:25.938363 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.938444 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.938600 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.938686 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.938848 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.938926 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.939082 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.939161 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.939326 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.939405 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.939560 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.939663 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.939830 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.939909 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.940077 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.940160 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.940324 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.940409 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.940552 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.940630 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.940796 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.940876 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.941026 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.941105 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.941268 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.941362 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.941502 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.941588 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.941735 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.941819 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.941983 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.942061 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.942221 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.942301 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.942464 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.942548 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.942699 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.942853 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.942935 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.943008 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.943174 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.943253 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.943429 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.943515 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.943717 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.943810 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.943976 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.944056 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.944217 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.944308 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.944448 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.944525 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.944681 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.944761 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.944909 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.944998 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.945144 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.945224 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.945379 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.945462 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.945608 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.945695 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.945846 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.945923 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.946077 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.946156 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.946313 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.946399 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.946560 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.946643 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.946788 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.946866 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.947022 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.947109 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.947251 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.947329 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.947476 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.947554 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.947730 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.947825 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.947962 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.948039 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.948192 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.948270 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.948421 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.948573 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.948655 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.948729 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.948877 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.948958 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.949111 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.949204 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.949343 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.949432 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.949575 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.949659 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.949804 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.949894 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.950040 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.950186 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.950285 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.950380 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.950476 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.950582 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.950708 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.950815 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.950945 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.951054 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.951177 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.951257 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.951409 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.951490 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.951700 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.951794 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.951925 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.952004 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.952124 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.952201 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.952361 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.952445 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.952592 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.952674 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.952807 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.952885 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.953043 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.953137 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.953288 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.953369 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.953525 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.953606 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.953754 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.953833 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.953992 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.954072 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.954228 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.954350 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.954475 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.954563 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.954719 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.954799 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.954953 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.955044 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.955188 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.955311 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.955420 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.955499 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.955669 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.955754 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.955908 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.955997 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.956143 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.956268 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.956377 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.956464 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.956609 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.956687 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.956843 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.956922 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.957075 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.957157 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.957308 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.957389 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.957539 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.957617 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.957773 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.957853 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.958008 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.958096 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.958248 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.958328 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.958484 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.958566 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.958726 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.958804 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.958959 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.959037 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.959195 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.959282 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.959445 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.959525 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.959673 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.959755 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.959908 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.959985 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.960137 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.960218 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.960352 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.960441 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.960577 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.960667 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.960810 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.960890 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.961040 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.961119 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.961278 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.961364 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.961511 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.961590 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.961757 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.961835 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.961995 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.962083 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.962237 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.962315 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.962467 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] auth.external = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.962558 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.962697 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.962784 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.962942 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] auth.external = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.963020 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.963173 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.963279 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.963420 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] auth.password = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.963501 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.963687 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] auth.token = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.963783 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.963915 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.963996 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] auth.password = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.964149 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.964229 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] auth.token = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.964392 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.964479 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.964620 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.964699 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.964866 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.964946 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.965101 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.965193 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.965328 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.965408 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.965559 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] credential.caching = True {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.965645 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.965805 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.965893 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.966038 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] credential.caching = True {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.966117 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.966286 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.966369 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.966521 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.966610 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.966764 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.966844 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.966995 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.967075 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.967236 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.967323 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.967472 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.967552 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.967730 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.967814 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.968515 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.968617 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.968762 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.968842 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.969017 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.969098 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.969276 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.969363 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.969520 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.969600 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.969777 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.969858 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.970009 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] federation.caching = True {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.970097 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.970244 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.970324 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.970485 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] federation.caching = True {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.970563 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.970723 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.970819 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.970948 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.971028 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.971194 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.971274 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.971421 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.971501 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.971685 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.971769 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.971923 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.972013 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.972174 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.972255 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.972407 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.972494 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.972655 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.972735 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.972890 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.972970 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] identity.caching = True {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.973133 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.973212 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.973365 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] identity.caching = True {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.973443 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.973597 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.973678 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.973829 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.973916 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.974069 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.974149 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.974301 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.974377 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.974519 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.974710 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.974792 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.974872 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.974981 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.975064 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.975217 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.975304 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.975448 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.975528 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.975699 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.975785 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.975934 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.976013 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.976169 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.976249 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.976411 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.976492 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.976654 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.976733 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.976897 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.976976 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.977148 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.977229 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.977387 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.977466 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.977622 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.977702 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.977850 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.977931 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.978083 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.978163 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.978313 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.978393 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.978545 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.978627 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.978775 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.978855 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.979009 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.979090 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.979249 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.979331 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.979474 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.979553 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.979748 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.979847 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.979983 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.980064 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.980213 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.980292 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.980458 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.980547 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.980690 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.980769 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.980918 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.980997 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.981154 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.981242 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.981382 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.981463 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.981616 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.981695 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.981845 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.981932 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.982079 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.982161 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.982311 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.982391 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.982545 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.982644 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.982780 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.982866 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.983018 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.983099 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.983244 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.983331 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.983472 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.983553 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.984319 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.984394 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.984523 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.984618 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.984772 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.984853 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.985008 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.985088 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.985240 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.985368 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.985452 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.user = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.985575 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.985692 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.985833 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.985912 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.user = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.986057 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.986139 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.986289 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.986500 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.986581 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.986739 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.986828 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.986991 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.987072 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.987475 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.987555 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.987731 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.987841 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.987998 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.988132 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.988210 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.988283 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.988419 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.988509 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.988648 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.988729 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.988884 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.988963 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.989124 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.989259 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.989346 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.989421 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.989594 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.989685 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.989838 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.989939 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.990066 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.990160 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.990310 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.990392 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.990539 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.990618 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.990770 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.990850 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.991003 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.991084 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.991235 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.991324 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.991469 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.991548 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.991727 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.991810 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.991964 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.992056 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.992212 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.992291 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.992437 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.992516 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.992678 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.992766 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.992911 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.992990 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.993139 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.993218 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.993372 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.993463 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.993636 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.993723 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.993841 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.993921 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.994071 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.994163 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.994311 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.994392 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.994540 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.994619 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] resource.caching = True {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.994779 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.994868 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.995019 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] resource.caching = True {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.995100 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.995250 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.995329 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.995497 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.995592 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.995790 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.995870 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.995987 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.996067 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.996232 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.996315 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.996465 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.996545 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.996700 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.996779 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.996930 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] role.caching = True {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.997012 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.997167 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.997246 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] role.driver = sql {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.997393 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.997471 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] role.caching = True {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.997626 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] role.driver = sql {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.997703 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.997856 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.997938 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.998108 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.998189 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.998350 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.998431 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.998593 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.998674 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.998836 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.998916 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.999079 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.999179 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.999347 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.999427 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.999575 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.999674 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.999844 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:25.999927 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.000082 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.000162 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.000324 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.000403 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.000569 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.000651 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.000814 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.000956 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.001064 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.001190 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.001438 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.001519 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.001680 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.001759 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.001921 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.002000 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.002166 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.002248 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.002398 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.002478 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.002645 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.002731 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.002881 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.002961 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.003114 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.003193 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.003346 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.003434 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.003577 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.003677 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.003832 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.003914 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.004074 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.004164 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.004308 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.004389 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.004560 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.004642 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.004790 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.004877 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.005023 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.005104 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.005251 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.005332 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.005482 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] token.caching = True {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.005571 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.005718 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.005797 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.005950 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] token.caching = True {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.006031 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.006180 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.006267 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.006424 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.006503 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.006659 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.006739 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.006888 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.006975 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.007127 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.007208 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.007364 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.007444 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.007603 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.007788 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.007856 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.007916 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.008084 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.008165 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.008313 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.008402 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.008556 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.008635 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.008789 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.008869 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.009015 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.009105 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.009253 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.009338 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.009487 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.009566 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.009727 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.009817 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] database.connection = **** {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.010013 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.010101 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.010257 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.010338 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] database.connection = **** {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.010492 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.010629 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.010731 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.010813 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.010963 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.011045 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.011197 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.011278 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.011429 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.011509 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.011673 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.011758 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.011917 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.011998 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.012149 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.012229 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.012390 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.012473 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.012622 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.012702 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.012863 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.012943 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.013094 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.013175 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.013335 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.013416 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.013563 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.013644 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.013799 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.013882 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.014028 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.014107 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.014180 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 22 19:04:26.014253 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.014489 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 19:04:26.014659 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 22 19:04:26.043448 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 19:04:26.043699 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 19:04:26.043787 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 19:04:26.043929 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 19:04:26.044016 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 19:04:26.044227 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 19:04:26.052877 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 19:04:26.053088 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 19:04:26.080956 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 19:04:26.081185 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 19:04:26.081309 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 19:04:26.081717 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 19:04:26.134513 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 19:04:26.134745 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 19:04:26.183260 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 19:04:26.183493 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 19:04:26.183652 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 19:04:26.183898 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 19:04:26.340348 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 19:04:26.340572 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 19:04:26.340794 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 19:04:26.340882 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 19:04:26.341013 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 19:04:26.341112 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 19:04:26.341230 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 19:04:26.341315 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 19:04:26.341442 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 19:04:26.341526 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 19:04:26.341691 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 19:04:26.341916 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 19:04:26.365197 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 19:04:26.365438 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 19:04:26.365661 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 19:04:26.365749 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 19:04:26.365889 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 19:04:26.365972 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 19:04:26.366106 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 19:04:26.366191 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 19:04:26.366319 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 19:04:26.366411 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 19:04:26.366533 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 19:04:26.366617 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 19:04:26.366743 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 19:04:26.366954 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 19:04:26.377698 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 19:04:26.377905 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 19:04:26.390963 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 19:04:26.391102 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 19:04:26.391189 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 19:04:26.391333 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 19:04:26.391420 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 19:04:26.391552 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 19:04:26.391636 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 19:04:26.391793 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 19:04:26.391876 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 19:04:26.392004 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 19:04:26.392089 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 19:04:26.392211 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 19:04:26.392294 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 19:04:26.392418 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 19:04:26.409301 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.409513 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 22 19:04:26.409711 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.409855 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.409938 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 22 19:04:26.410055 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 22 19:04:26.410244 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.410328 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.410445 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 22 19:04:26.410573 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.410719 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.410982 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.411084 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.411352 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.411483 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.411770 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.420685 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.420964 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.421048 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.421262 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.421346 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.421547 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.421640 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.421842 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.428493 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81532) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 19:04:26.428734 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81533) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 19:04:26.431372 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81532) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 19:04:26.431581 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81533) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 19:04:26.431693 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.431892 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.431975 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.432177 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.432260 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.432459 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.432544 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.432734 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.432815 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.432992 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.433081 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.433261 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.433344 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.433534 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.433638 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.433911 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.433995 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.434179 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.434301 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.434446 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.457270 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.457433 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.458723 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81532) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 19:04:26.458876 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81533) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 19:04:26.459012 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.459161 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.463798 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81532) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 19:04:26.463936 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81533) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 19:04:26.464086 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.464221 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.464368 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.464502 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.471322 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81532) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 19:04:26.471483 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81533) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 19:04:26.471611 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.471801 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.475800 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81532) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 19:04:26.475945 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81533) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 19:04:26.476096 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.476234 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.476390 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.476524 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.476684 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.476805 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.476962 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.477079 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.477248 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.477364 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.477521 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.477635 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.477801 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.477909 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.494924 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.495038 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.495204 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.495289 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.498996 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81533) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 19:04:26.499088 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81532) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 19:04:26.502662 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.502754 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.502955 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.503038 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.507073 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81533) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 19:04:26.507175 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81532) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 19:04:26.510150 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81532) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 19:04:26.510236 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81533) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 19:04:26.513314 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.513399 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.513613 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.513699 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.513897 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.513980 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.514180 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.514270 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.514455 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.514537 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.514732 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.514897 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.530965 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81533) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 19:04:26.531061 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81532) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 19:04:26.531251 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.531335 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.531519 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.531601 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.531805 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.531891 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.538610 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.538702 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.540149 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.540246 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.540425 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.540507 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.543022 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81533) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 19:04:26.543104 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81532) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 19:04:26.543314 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.543429 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.543604 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.543715 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.543917 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.543998 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.544194 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.544325 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.555545 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81532) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 19:04:26.555774 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81533) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 19:04:26.555861 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.556056 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.556142 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.556335 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.556418 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.556609 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.556702 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.556884 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.556969 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.557156 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.557240 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.557437 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.573060 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81532) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 19:04:26.573265 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81533) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 19:04:26.576107 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81532) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 19:04:26.576316 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81533) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 19:04:26.579528 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.579811 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.580991 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.581242 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.582463 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81532) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 19:04:26.582697 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81533) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 19:04:26.582782 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.582986 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.583069 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.583275 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.589502 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.589722 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.590946 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81532) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 19:04:26.591161 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81533) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 19:04:26.593914 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.594210 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.594309 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.594495 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.594578 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.594772 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.594854 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.595051 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.602659 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81532) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 19:04:26.602916 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81533) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 19:04:26.603001 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.603205 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.603288 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.603480 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.610102 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81532) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 19:04:26.610323 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81533) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 19:04:26.610407 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.610608 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.610700 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.610886 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.610969 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.611156 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.611238 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.611422 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.611512 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.611712 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.611796 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.611984 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.612065 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.612511 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.612597 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.612785 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.612866 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.613048 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.613137 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.613305 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.613394 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.613567 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.613689 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.613842 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.613923 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.614105 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 19:04:26.648570 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55d7dda8e9c0 pid: 81532 (default app) Aug 22 19:04:26.648669 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55d7dda8e9c0 pid: 81533 (default app) Aug 22 19:04:26.651913 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679b41f7-f614-468c-a43e-256572d7cd7d None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:26.652200 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679b41f7-f614-468c-a43e-256572d7cd7d None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:26.652473 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679b41f7-f614-468c-a43e-256572d7cd7d None None] PATH_INFO: `/v3` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:26.656661 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1/1] 10.222.0.61 () {50 vars in 791 bytes} [Mon Aug 22 19:04:26 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:04:33.165422 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b76167a-5568-4d8a-b54c-87b23b43327c None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:33.165851 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b76167a-5568-4d8a-b54c-87b23b43327c None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:33.165965 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b76167a-5568-4d8a-b54c-87b23b43327c None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:33.170187 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1/2] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:04:33 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 19:04:33.176994 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8277301a-2edc-4cf8-83fa-c1deb9c2b317 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:33.177414 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8277301a-2edc-4cf8-83fa-c1deb9c2b317 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:33.177536 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8277301a-2edc-4cf8-83fa-c1deb9c2b317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:33.247846 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: INFO dbcounter [None req-8277301a-2edc-4cf8-83fa-c1deb9c2b317 None None] Registered counter for database keystone Aug 22 19:04:33.266546 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG oslo_db.sqlalchemy.engines [None req-8277301a-2edc-4cf8-83fa-c1deb9c2b317 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81532) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 22 19:04:33.272932 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writer thread running {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 22 19:04:33.452209 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG passlib.handlers.bcrypt [None req-8277301a-2edc-4cf8-83fa-c1deb9c2b317 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81532) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 22 19:04:33.453267 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG passlib.handlers.bcrypt [None req-8277301a-2edc-4cf8-83fa-c1deb9c2b317 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81532) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 22 19:04:33.508949 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-8277301a-2edc-4cf8-83fa-c1deb9c2b317 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:04:33.510561 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-8277301a-2edc-4cf8-83fa-c1deb9c2b317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:04:33.638783 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 2/3] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:04:33 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 463 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 22 19:04:34.062548 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b76167a-5568-4d8a-b54c-87b23b43327c None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:04:34.130665 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: INFO dbcounter [None req-7b76167a-5568-4d8a-b54c-87b23b43327c None None] Registered counter for database keystone Aug 22 19:04:34.152513 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG oslo_db.sqlalchemy.engines [None req-7b76167a-5568-4d8a-b54c-87b23b43327c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81533) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 22 19:04:34.158656 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writer thread running {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 22 19:04:34.277938 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b76167a-5568-4d8a-b54c-87b23b43327c None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:04:34.282240 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66df2572-bd00-44ac-9422-6fc26056ccf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:04:34.282868 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66df2572-bd00-44ac-9422-6fc26056ccf7 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:34.283147 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66df2572-bd00-44ac-9422-6fc26056ccf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:34.283419 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66df2572-bd00-44ac-9422-6fc26056ccf7 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:34.291452 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66df2572-bd00-44ac-9422-6fc26056ccf7 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:04:34.342563 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: WARNING keystone.common.rbac_enforcer.enforcer [None req-66df2572-bd00-44ac-9422-6fc26056ccf7 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 22 19:04:34.342852 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66df2572-bd00-44ac-9422-6fc26056ccf7 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:04:34.352510 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 2/4] 10.222.0.61 () {58 vars in 1324 bytes} [Mon Aug 22 19:04:34 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 22 19:04:34.358416 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8277301a-2edc-4cf8-83fa-c1deb9c2b317 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:04:34.373429 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8277301a-2edc-4cf8-83fa-c1deb9c2b317 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:04:34.378408 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46995fdc-d977-4380-b165-c28e78b6c8b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:04:34.379519 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46995fdc-d977-4380-b165-c28e78b6c8b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:34.379831 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46995fdc-d977-4380-b165-c28e78b6c8b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:34.380104 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46995fdc-d977-4380-b165-c28e78b6c8b8 admin admin] PATH_INFO: `/v3/projects/4bc536d83f9841a59997d0d04e993cd8` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:34.381663 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46995fdc-d977-4380-b165-c28e78b6c8b8 admin admin] RBAC: Authorizing `identity:get_project(project_id=4bc536d83f9841a59997d0d04e993cd8)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:04:34.433622 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: WARNING keystone.common.rbac_enforcer.enforcer [None req-46995fdc-d977-4380-b165-c28e78b6c8b8 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 22 19:04:34.433964 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46995fdc-d977-4380-b165-c28e78b6c8b8 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:04:34.439071 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 3/5] 10.222.0.61 () {56 vars in 1207 bytes} [Mon Aug 22 19:04:34 2022] GET /identity/v3/projects/4bc536d83f9841a59997d0d04e993cd8 => generated 344 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:04:36.290039 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a9aa7e-01f3-4ccb-849b-814a89dd3725 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:36.290598 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a9aa7e-01f3-4ccb-849b-814a89dd3725 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:36.290598 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a9aa7e-01f3-4ccb-849b-814a89dd3725 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:36.294710 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 3/6] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:04:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 19:04:36.300844 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53fa03b-5b56-4abe-8501-47f21751a1b9 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:36.301289 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53fa03b-5b56-4abe-8501-47f21751a1b9 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:36.301383 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53fa03b-5b56-4abe-8501-47f21751a1b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:36.320859 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-c53fa03b-5b56-4abe-8501-47f21751a1b9 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:04:36.322379 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-c53fa03b-5b56-4abe-8501-47f21751a1b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:04:36.330160 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 4/7] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:04:36 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 22 19:04:36.602772 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a9aa7e-01f3-4ccb-849b-814a89dd3725 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:04:36.615323 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a9aa7e-01f3-4ccb-849b-814a89dd3725 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:04:36.619552 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d08768d-0d36-4e6d-8ce0-0dd32d64fffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:04:36.620182 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d08768d-0d36-4e6d-8ce0-0dd32d64fffd admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:36.620463 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d08768d-0d36-4e6d-8ce0-0dd32d64fffd admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:36.620736 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d08768d-0d36-4e6d-8ce0-0dd32d64fffd admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:36.628464 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d08768d-0d36-4e6d-8ce0-0dd32d64fffd admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:04:36.630245 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d08768d-0d36-4e6d-8ce0-0dd32d64fffd admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:04:36.639246 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 4/8] 10.222.0.61 () {58 vars in 1324 bytes} [Mon Aug 22 19:04:36 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 22 19:04:36.644967 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53fa03b-5b56-4abe-8501-47f21751a1b9 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:04:36.653972 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53fa03b-5b56-4abe-8501-47f21751a1b9 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:04:36.658864 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6498f0-3963-46d6-ba08-4351d1f73f96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:04:36.659484 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6498f0-3963-46d6-ba08-4351d1f73f96 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:36.659799 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6498f0-3963-46d6-ba08-4351d1f73f96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:36.660070 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6498f0-3963-46d6-ba08-4351d1f73f96 admin admin] PATH_INFO: `/v3/users/d9c7158eb3ce41ab8bb7a5d05da4a2f6` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:36.661710 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6498f0-3963-46d6-ba08-4351d1f73f96 admin admin] RBAC: Authorizing `identity:get_user(user_id=d9c7158eb3ce41ab8bb7a5d05da4a2f6)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:04:36.663513 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6498f0-3963-46d6-ba08-4351d1f73f96 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:04:36.665076 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 5/9] 10.222.0.61 () {56 vars in 1198 bytes} [Mon Aug 22 19:04:36 2022] GET /identity/v3/users/d9c7158eb3ce41ab8bb7a5d05da4a2f6 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:04:38.636982 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f82f180-93f1-4d7a-b8fe-945414144907 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:38.637539 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f82f180-93f1-4d7a-b8fe-945414144907 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:38.637539 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f82f180-93f1-4d7a-b8fe-945414144907 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:38.639166 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 5/10] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:04:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:04:38.639596 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47b11d1-5c2b-4b2c-9a9c-aed00f981192 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:38.640007 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47b11d1-5c2b-4b2c-9a9c-aed00f981192 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:38.640230 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47b11d1-5c2b-4b2c-9a9c-aed00f981192 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:38.641663 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 6/11] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:04:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 19:04:38.643986 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13442a9c-a709-4f08-937c-f3e6a04a485c None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:38.644277 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13442a9c-a709-4f08-937c-f3e6a04a485c None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:38.644559 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13442a9c-a709-4f08-937c-f3e6a04a485c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:38.646683 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8c4fd1-78fc-4dfe-b040-928d064ee555 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:38.646969 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8c4fd1-78fc-4dfe-b040-928d064ee555 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:38.647245 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8c4fd1-78fc-4dfe-b040-928d064ee555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:38.665484 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-9e8c4fd1-78fc-4dfe-b040-928d064ee555 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:04:38.666670 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-9e8c4fd1-78fc-4dfe-b040-928d064ee555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:04:38.687427 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG passlib.handlers.bcrypt [None req-13442a9c-a709-4f08-937c-f3e6a04a485c None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81533) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 22 19:04:38.687963 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG passlib.handlers.bcrypt [None req-13442a9c-a709-4f08-937c-f3e6a04a485c None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81533) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 22 19:04:38.733362 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-13442a9c-a709-4f08-937c-f3e6a04a485c None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:04:38.735511 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-13442a9c-a709-4f08-937c-f3e6a04a485c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:04:38.748312 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 7/12] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:04:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 19:04:38.832881 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 6/13] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:04:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 19:04:39.013303 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8c4fd1-78fc-4dfe-b040-928d064ee555 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:04:39.038189 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:04:39.096396 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13442a9c-a709-4f08-937c-f3e6a04a485c None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:04:39.122323 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8c4fd1-78fc-4dfe-b040-928d064ee555 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:04:39.156399 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6082824-3592-4010-93a2-22bc4a05f851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:04:39.157027 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6082824-3592-4010-93a2-22bc4a05f851 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:39.157307 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6082824-3592-4010-93a2-22bc4a05f851 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:39.157576 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6082824-3592-4010-93a2-22bc4a05f851 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:39.205211 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13442a9c-a709-4f08-937c-f3e6a04a485c None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:04:39.207487 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:04:39.225312 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6082824-3592-4010-93a2-22bc4a05f851 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:04:39.227046 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6082824-3592-4010-93a2-22bc4a05f851 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:04:39.244464 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05e4034-5dc2-4940-8dd1-ae6a36a073e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:04:39.245402 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e4034-5dc2-4940-8dd1-ae6a36a073e6 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:39.245687 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e4034-5dc2-4940-8dd1-ae6a36a073e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:39.245962 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e4034-5dc2-4940-8dd1-ae6a36a073e6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:39.250089 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:04:39.258379 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05e4034-5dc2-4940-8dd1-ae6a36a073e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 19:04:39.267359 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-f05e4034-5dc2-4940-8dd1-ae6a36a073e6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 19:04:39.267359 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:04:39.267359 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:04:39.267359 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:04:39.267359 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:04:39.267359 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:04:39.267359 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:04:39.267359 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:04:39.267359 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:04:39.267359 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:04:39.267359 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:04:39.267359 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:04:39.267359 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 19:04:39.267359 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 19:04:39.267359 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 19:04:39.267359 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 19:04:39.268475 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 19:04:39.268475 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 19:04:39.268475 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 19:04:39.268475 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 19:04:39.268475 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:04:39.268475 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:04:39.268475 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:04:39.268475 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:04:39.268475 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:04:39.268475 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:04:39.268475 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:04:39.268475 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:04:39.268475 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:04:39.268475 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:04:39.268475 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:04:39.268475 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:04:39.269467 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:04:39.269467 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:04:39.269467 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:04:39.269467 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:04:39.269467 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 19:04:39.269467 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 19:04:39.269467 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 19:04:39.269467 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 19:04:39.269467 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 19:04:39.269467 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 19:04:39.269467 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 19:04:39.269467 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:04:39.269467 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 7/14] 10.222.0.61 () {56 vars in 1096 bytes} [Mon Aug 22 19:04:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 19:04:39.274143 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05e4034-5dc2-4940-8dd1-ae6a36a073e6 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:04:39.325859 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 8/15] 10.222.0.61 () {58 vars in 1282 bytes} [Mon Aug 22 19:04:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 22 19:04:39.331877 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6082824-3592-4010-93a2-22bc4a05f851 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:04:39.372507 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05e4034-5dc2-4940-8dd1-ae6a36a073e6 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:04:39.406187 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca347f0-b4da-4385-a46c-63d98961b953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:04:39.406795 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca347f0-b4da-4385-a46c-63d98961b953 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:39.407066 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca347f0-b4da-4385-a46c-63d98961b953 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:39.407332 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca347f0-b4da-4385-a46c-63d98961b953 None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:39.408034 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca347f0-b4da-4385-a46c-63d98961b953 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:04:39.408328 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca347f0-b4da-4385-a46c-63d98961b953 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:04:39.409750 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca347f0-b4da-4385-a46c-63d98961b953 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:04:39.421298 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 8/16] 10.222.0.61 () {56 vars in 1099 bytes} [Mon Aug 22 19:04:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 19:04:39.426860 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca347f0-b4da-4385-a46c-63d98961b953 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:04:39.429111 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6082824-3592-4010-93a2-22bc4a05f851 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:04:39.451707 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:04:39.454275 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:04:39.464150 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc0a44d-4b6e-4ad3-a0ec-d4255d3914fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:04:39.464783 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc0a44d-4b6e-4ad3-a0ec-d4255d3914fc None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:39.465064 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc0a44d-4b6e-4ad3-a0ec-d4255d3914fc None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:39.465335 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc0a44d-4b6e-4ad3-a0ec-d4255d3914fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:39.475165 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc0a44d-4b6e-4ad3-a0ec-d4255d3914fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:04:39.476656 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc0a44d-4b6e-4ad3-a0ec-d4255d3914fc None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:04:39.494545 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-0cc0a44d-4b6e-4ad3-a0ec-d4255d3914fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:04:39.494545 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:04:39.494545 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:04:39.494545 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:04:39.494545 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:04:39.494545 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:04:39.494545 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:04:39.494545 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:04:39.494545 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:04:39.494545 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:04:39.494545 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:04:39.494545 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:04:39.494545 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 19:04:39.494545 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 19:04:39.494545 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 19:04:39.494545 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 19:04:39.495483 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:04:39.495483 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:04:39.495483 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:04:39.495483 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:04:39.495483 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:04:39.495483 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:04:39.495483 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:04:39.495483 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:04:39.495483 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:04:39.495483 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:04:39.495483 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:04:39.495483 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:04:39.495483 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:04:39.495483 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:04:39.495483 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:04:39.495483 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:04:39.495483 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 19:04:39.496359 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 19:04:39.496359 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:04:39.496359 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:04:39.496359 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 9/17] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:04:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 19:04:39.501902 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc0a44d-4b6e-4ad3-a0ec-d4255d3914fc None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:04:39.530880 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca347f0-b4da-4385-a46c-63d98961b953 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:04:39.564237 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212bd2e6-5894-4b1d-8559-10697411a831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:04:39.564868 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212bd2e6-5894-4b1d-8559-10697411a831 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:39.565147 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212bd2e6-5894-4b1d-8559-10697411a831 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:39.565420 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212bd2e6-5894-4b1d-8559-10697411a831 None admin] PATH_INFO: `/v3/users/d9c7158eb3ce41ab8bb7a5d05da4a2f6` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:39.566899 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212bd2e6-5894-4b1d-8559-10697411a831 None admin] RBAC: Authorizing `identity:get_user(user_id=d9c7158eb3ce41ab8bb7a5d05da4a2f6)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:04:39.568390 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212bd2e6-5894-4b1d-8559-10697411a831 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:04:39.569971 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 9/18] 10.222.0.61 () {56 vars in 1177 bytes} [Mon Aug 22 19:04:39 2022] GET /identity/v3/users/d9c7158eb3ce41ab8bb7a5d05da4a2f6 => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 19:04:39.575397 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212bd2e6-5894-4b1d-8559-10697411a831 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:04:39.601246 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc0a44d-4b6e-4ad3-a0ec-d4255d3914fc None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:04:39.635848 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd444b73-b865-49d3-88b6-c6b297d6f4c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:04:39.636448 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd444b73-b865-49d3-88b6-c6b297d6f4c4 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:39.636721 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd444b73-b865-49d3-88b6-c6b297d6f4c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:39.636988 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd444b73-b865-49d3-88b6-c6b297d6f4c4 None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:39.637620 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd444b73-b865-49d3-88b6-c6b297d6f4c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:04:39.637917 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd444b73-b865-49d3-88b6-c6b297d6f4c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:04:39.639280 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd444b73-b865-49d3-88b6-c6b297d6f4c4 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:04:39.657077 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 10/19] 10.222.0.61 () {56 vars in 1109 bytes} [Mon Aug 22 19:04:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 19:04:39.667620 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:04:39.675840 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212bd2e6-5894-4b1d-8559-10697411a831 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:04:39.712145 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0149d34b-e15e-46ec-b688-695068cb0219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:04:39.712794 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0149d34b-e15e-46ec-b688-695068cb0219 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:39.713069 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0149d34b-e15e-46ec-b688-695068cb0219 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:39.713337 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0149d34b-e15e-46ec-b688-695068cb0219 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:39.714582 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0149d34b-e15e-46ec-b688-695068cb0219 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:04:39.716102 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0149d34b-e15e-46ec-b688-695068cb0219 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:04:39.717746 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 10/20] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:04:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 19:04:39.723927 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd444b73-b865-49d3-88b6-c6b297d6f4c4 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:04:39.761371 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:04:39.830409 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd444b73-b865-49d3-88b6-c6b297d6f4c4 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:04:39.865269 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde38c93-49c1-4a2b-a591-fe2d9f9a7d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:04:39.865895 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde38c93-49c1-4a2b-a591-fe2d9f9a7d6e None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:39.866175 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde38c93-49c1-4a2b-a591-fe2d9f9a7d6e None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:39.866447 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde38c93-49c1-4a2b-a591-fe2d9f9a7d6e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:39.867188 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde38c93-49c1-4a2b-a591-fe2d9f9a7d6e None admin] RBAC: Adding query filter params (role.id=163c30c3b3c246beb849ce176f02d9da, scope.domain.id=default, user.id=d9c7158eb3ce41ab8bb7a5d05da4a2f6) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:04:39.867494 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde38c93-49c1-4a2b-a591-fe2d9f9a7d6e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:04:39.869025 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde38c93-49c1-4a2b-a591-fe2d9f9a7d6e None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:04:39.876696 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 11/21] 10.222.0.61 () {56 vars in 1322 bytes} [Mon Aug 22 19:04:39 2022] GET /identity/v3/role_assignments?user.id=d9c7158eb3ce41ab8bb7a5d05da4a2f6&scope.domain.id=default&role.id=163c30c3b3c246beb849ce176f02d9da => generated 233 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:04:41.867285 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f130432-ba9b-4d3b-8ec5-5e43e81fc9fa None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:41.868321 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f130432-ba9b-4d3b-8ec5-5e43e81fc9fa None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:41.868796 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f130432-ba9b-4d3b-8ec5-5e43e81fc9fa None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:41.870634 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 11/22] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:04:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:04:41.876252 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c5ab22-6fc2-4821-ac86-9bfa15b4c41a None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:41.877072 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c5ab22-6fc2-4821-ac86-9bfa15b4c41a None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:41.877548 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c5ab22-6fc2-4821-ac86-9bfa15b4c41a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:41.904644 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-70c5ab22-6fc2-4821-ac86-9bfa15b4c41a None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:04:41.906233 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-70c5ab22-6fc2-4821-ac86-9bfa15b4c41a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:04:41.983671 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 12/23] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:04:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 19:04:42.090759 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccd16bd-225e-49d1-a96d-01ee2e5f5c79 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:42.091383 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccd16bd-225e-49d1-a96d-01ee2e5f5c79 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:42.091903 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccd16bd-225e-49d1-a96d-01ee2e5f5c79 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:42.093734 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 12/24] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:04:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:04:42.104986 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829025b9-8d97-4819-af66-00a6cb73f918 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:42.105278 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829025b9-8d97-4819-af66-00a6cb73f918 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:42.105556 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829025b9-8d97-4819-af66-00a6cb73f918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:42.132720 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-829025b9-8d97-4819-af66-00a6cb73f918 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:04:42.133874 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-829025b9-8d97-4819-af66-00a6cb73f918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:04:42.162767 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:04:42.219848 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 13/25] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:04:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 19:04:42.225287 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014e2df7-404c-43e0-b7fb-023f0c04bc68 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:42.225796 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014e2df7-404c-43e0-b7fb-023f0c04bc68 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:42.226285 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014e2df7-404c-43e0-b7fb-023f0c04bc68 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:42.227846 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 13/26] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:04:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:04:42.232991 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c431655-e10a-42d8-9927-d3fda4599ea5 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:42.233285 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c431655-e10a-42d8-9927-d3fda4599ea5 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:42.233566 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c431655-e10a-42d8-9927-d3fda4599ea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:42.259666 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-7c431655-e10a-42d8-9927-d3fda4599ea5 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:04:42.260882 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-7c431655-e10a-42d8-9927-d3fda4599ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:04:42.349003 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 14/27] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:04:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 19:04:42.416175 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014e2df7-404c-43e0-b7fb-023f0c04bc68 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:04:42.565392 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014e2df7-404c-43e0-b7fb-023f0c04bc68 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:04:42.584999 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c431655-e10a-42d8-9927-d3fda4599ea5 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:04:42.619471 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d2a081-3fa7-4415-bfe0-d003eeaeacaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:04:42.620510 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d2a081-3fa7-4415-bfe0-d003eeaeacaa None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:42.621021 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d2a081-3fa7-4415-bfe0-d003eeaeacaa None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:42.621786 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d2a081-3fa7-4415-bfe0-d003eeaeacaa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:42.650108 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d2a081-3fa7-4415-bfe0-d003eeaeacaa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:04:42.651974 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d2a081-3fa7-4415-bfe0-d003eeaeacaa None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:04:42.680383 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-27d2a081-3fa7-4415-bfe0-d003eeaeacaa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:04:42.680383 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:04:42.680383 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:04:42.680383 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:04:42.680383 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:04:42.680383 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:04:42.680383 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:04:42.680383 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:04:42.680383 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:04:42.680383 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:04:42.680383 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:04:42.680383 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:04:42.680383 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 19:04:42.680383 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 19:04:42.680383 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 19:04:42.680383 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 19:04:42.681289 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:04:42.681289 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:04:42.681289 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:04:42.681289 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:04:42.681289 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:04:42.681289 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:04:42.681289 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:04:42.681289 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:04:42.681289 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:04:42.681289 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:04:42.681289 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:04:42.681289 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:04:42.681289 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:04:42.681289 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:04:42.681289 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:04:42.681289 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:04:42.681289 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 19:04:42.682094 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 19:04:42.682094 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:04:42.682094 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:04:42.689127 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 14/28] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:04:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:04:42.690753 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961b3200-43dd-4dc3-abb4-902afe54b6c1 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:42.691352 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961b3200-43dd-4dc3-abb4-902afe54b6c1 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:42.691874 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961b3200-43dd-4dc3-abb4-902afe54b6c1 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:42.693361 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 15/29] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:04:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:04:42.695602 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961b3200-43dd-4dc3-abb4-902afe54b6c1 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:04:42.722472 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:04:42.725070 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:04:42.748411 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c431655-e10a-42d8-9927-d3fda4599ea5 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:04:42.788255 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288fdb5c-756f-4da3-b56a-bb2f989356ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:04:42.788923 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288fdb5c-756f-4da3-b56a-bb2f989356ac None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:42.789201 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288fdb5c-756f-4da3-b56a-bb2f989356ac None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:42.789474 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288fdb5c-756f-4da3-b56a-bb2f989356ac None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:42.801463 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288fdb5c-756f-4da3-b56a-bb2f989356ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 19:04:42.808267 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-288fdb5c-756f-4da3-b56a-bb2f989356ac None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 19:04:42.808267 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:04:42.808267 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:04:42.808267 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:04:42.808267 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:04:42.808267 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:04:42.808267 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:04:42.808267 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:04:42.808267 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:04:42.808267 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:04:42.808267 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:04:42.808267 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:04:42.808267 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 19:04:42.808267 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 19:04:42.808267 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 19:04:42.808267 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 19:04:42.809142 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 19:04:42.809142 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 19:04:42.809142 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 19:04:42.809142 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 19:04:42.809142 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:04:42.809142 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:04:42.809142 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:04:42.809142 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:04:42.809142 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:04:42.809142 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:04:42.809142 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:04:42.809142 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:04:42.809142 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:04:42.809142 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:04:42.809142 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:04:42.809142 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:04:42.810509 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:04:42.810509 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:04:42.810509 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:04:42.810509 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:04:42.810509 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 19:04:42.810509 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 19:04:42.810509 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 19:04:42.810509 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 19:04:42.810509 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 19:04:42.810509 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 19:04:42.810509 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 19:04:42.810509 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:04:42.810509 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 15/30] 10.222.0.61 () {56 vars in 1096 bytes} [Mon Aug 22 19:04:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 19:04:42.810509 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999fe47c-36fc-4d3c-999c-24d5cc001553 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:42.811202 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999fe47c-36fc-4d3c-999c-24d5cc001553 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:42.811202 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999fe47c-36fc-4d3c-999c-24d5cc001553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:42.837037 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961b3200-43dd-4dc3-abb4-902afe54b6c1 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:04:42.838109 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-999fe47c-36fc-4d3c-999c-24d5cc001553 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:04:42.839727 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-999fe47c-36fc-4d3c-999c-24d5cc001553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:04:42.875075 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b314a1-dd3e-4456-8b24-c700dbebfdb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:04:42.876376 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b314a1-dd3e-4456-8b24-c700dbebfdb2 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:42.876716 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b314a1-dd3e-4456-8b24-c700dbebfdb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:42.877059 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b314a1-dd3e-4456-8b24-c700dbebfdb2 None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:42.877775 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b314a1-dd3e-4456-8b24-c700dbebfdb2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:04:42.878163 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b314a1-dd3e-4456-8b24-c700dbebfdb2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:04:42.879629 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b314a1-dd3e-4456-8b24-c700dbebfdb2 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:04:42.904001 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 16/31] 10.222.0.61 () {56 vars in 1109 bytes} [Mon Aug 22 19:04:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:04:42.904001 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b314a1-dd3e-4456-8b24-c700dbebfdb2 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:04:42.934438 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 16/32] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:04:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 19:04:42.935606 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999fe47c-36fc-4d3c-999c-24d5cc001553 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:04:42.995076 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:04:43.020314 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b314a1-dd3e-4456-8b24-c700dbebfdb2 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:04:43.060951 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5385a2-3b84-4802-8dae-4bd8570c4e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:04:43.062613 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5385a2-3b84-4802-8dae-4bd8570c4e3f None admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:43.063095 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5385a2-3b84-4802-8dae-4bd8570c4e3f None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:43.063563 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5385a2-3b84-4802-8dae-4bd8570c4e3f None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:43.064632 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5385a2-3b84-4802-8dae-4bd8570c4e3f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:04:43.066484 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5385a2-3b84-4802-8dae-4bd8570c4e3f None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:04:43.105784 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:04:43.157812 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999fe47c-36fc-4d3c-999c-24d5cc001553 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:04:43.240898 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 17/33] 10.222.0.61 () {60 vars in 1131 bytes} [Mon Aug 22 19:04:42 2022] POST /identity/v3/roles => generated 225 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:04:43.242170 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5385a2-3b84-4802-8dae-4bd8570c4e3f None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:04:43.260422 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f26e0b1-616e-42da-ba2f-459733304de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:04:43.261330 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f26e0b1-616e-42da-ba2f-459733304de2 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:43.261755 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f26e0b1-616e-42da-ba2f-459733304de2 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:43.262180 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f26e0b1-616e-42da-ba2f-459733304de2 None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:43.263027 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f26e0b1-616e-42da-ba2f-459733304de2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:04:43.263491 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f26e0b1-616e-42da-ba2f-459733304de2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:04:43.265109 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f26e0b1-616e-42da-ba2f-459733304de2 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:04:43.288444 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 17/34] 10.222.0.61 () {56 vars in 1099 bytes} [Mon Aug 22 19:04:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:04:43.289520 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f26e0b1-616e-42da-ba2f-459733304de2 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:04:43.415873 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5385a2-3b84-4802-8dae-4bd8570c4e3f None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:04:43.462900 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40849818-bc13-46c2-89c3-426079acac60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:04:43.463987 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40849818-bc13-46c2-89c3-426079acac60 None admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:43.464541 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40849818-bc13-46c2-89c3-426079acac60 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:43.464824 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40849818-bc13-46c2-89c3-426079acac60 None admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:43.465732 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40849818-bc13-46c2-89c3-426079acac60 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:04:43.468360 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40849818-bc13-46c2-89c3-426079acac60 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:04:43.469292 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f26e0b1-616e-42da-ba2f-459733304de2 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:04:43.474939 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:04:43.510075 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 18/35] 10.222.0.61 () {60 vars in 1141 bytes} [Mon Aug 22 19:04:43 2022] POST /identity/v3/projects => generated 301 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:04:43.511134 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40849818-bc13-46c2-89c3-426079acac60 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:04:43.517602 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d31a421-d7a4-47c4-8ec2-a2323161d7a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:04:43.518588 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d31a421-d7a4-47c4-8ec2-a2323161d7a5 None admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:43.519065 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d31a421-d7a4-47c4-8ec2-a2323161d7a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:43.519584 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d31a421-d7a4-47c4-8ec2-a2323161d7a5 None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:43.520582 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d31a421-d7a4-47c4-8ec2-a2323161d7a5 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:04:43.522407 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d31a421-d7a4-47c4-8ec2-a2323161d7a5 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:04:43.539320 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:04:43.642433 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40849818-bc13-46c2-89c3-426079acac60 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:04:43.689095 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 18/36] 10.222.0.61 () {60 vars in 1131 bytes} [Mon Aug 22 19:04:43 2022] POST /identity/v3/roles => generated 219 bytes in 400 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:04:43.690525 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c9adc2-7a1b-4c94-8c6c-0306463fa275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:04:43.690834 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564cf3b5-e903-4aab-b831-d68251313ef6 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:43.691120 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564cf3b5-e903-4aab-b831-d68251313ef6 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:43.691227 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c9adc2-7a1b-4c94-8c6c-0306463fa275 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:43.691399 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564cf3b5-e903-4aab-b831-d68251313ef6 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:43.691495 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c9adc2-7a1b-4c94-8c6c-0306463fa275 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:43.691883 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c9adc2-7a1b-4c94-8c6c-0306463fa275 None admin] PATH_INFO: `/v3/users/d9c7158eb3ce41ab8bb7a5d05da4a2f6` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:43.692681 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 19/37] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:04:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:04:43.693582 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c9adc2-7a1b-4c94-8c6c-0306463fa275 None admin] RBAC: Authorizing `identity:get_user(user_id=d9c7158eb3ce41ab8bb7a5d05da4a2f6)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:04:43.693745 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efb9bd6-1584-4bcc-87e4-d44d9e44edfb None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:43.694009 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efb9bd6-1584-4bcc-87e4-d44d9e44edfb None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:43.694279 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efb9bd6-1584-4bcc-87e4-d44d9e44edfb None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:43.695090 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c9adc2-7a1b-4c94-8c6c-0306463fa275 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:04:43.695700 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 20/38] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:04:43 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:04:43.697368 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 19/39] 10.222.0.61 () {56 vars in 1177 bytes} [Mon Aug 22 19:04:43 2022] GET /identity/v3/users/d9c7158eb3ce41ab8bb7a5d05da4a2f6 => generated 247 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:04:43.702398 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7afe17-c137-49a4-98b8-21a1f904664b None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:43.702697 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7afe17-c137-49a4-98b8-21a1f904664b None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:43.703035 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7afe17-c137-49a4-98b8-21a1f904664b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:43.703421 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd8b15a-3088-46d7-a174-709a9e79c430 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:43.704003 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd8b15a-3088-46d7-a174-709a9e79c430 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:43.704485 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd8b15a-3088-46d7-a174-709a9e79c430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:43.723322 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-1d7afe17-c137-49a4-98b8-21a1f904664b None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:04:43.724933 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-1d7afe17-c137-49a4-98b8-21a1f904664b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:04:43.728155 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-5cd8b15a-3088-46d7-a174-709a9e79c430 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:04:43.729742 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-5cd8b15a-3088-46d7-a174-709a9e79c430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:04:43.790371 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:04:43.812529 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 20/40] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:04:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 19:04:43.813759 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd8b15a-3088-46d7-a174-709a9e79c430 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:04:43.826640 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 21/41] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:04:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 19:04:43.925370 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd8b15a-3088-46d7-a174-709a9e79c430 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:04:43.960277 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3205ab-4b08-4a42-9029-322bd75a8f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:04:43.961206 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3205ab-4b08-4a42-9029-322bd75a8f2f None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:43.961705 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3205ab-4b08-4a42-9029-322bd75a8f2f None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:43.962203 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3205ab-4b08-4a42-9029-322bd75a8f2f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:43.963550 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3205ab-4b08-4a42-9029-322bd75a8f2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:04:43.965199 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3205ab-4b08-4a42-9029-322bd75a8f2f None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:04:43.966904 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 21/42] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:04:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:04:43.973353 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7afe17-c137-49a4-98b8-21a1f904664b None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:04:44.089220 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3205ab-4b08-4a42-9029-322bd75a8f2f None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:04:44.146940 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:04:44.181146 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7afe17-c137-49a4-98b8-21a1f904664b None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:04:44.193925 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3205ab-4b08-4a42-9029-322bd75a8f2f None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:04:44.209188 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:04:44.214994 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb16172-2fb8-4cb1-a5d7-743ea26859c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:04:44.215673 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb16172-2fb8-4cb1-a5d7-743ea26859c1 None admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:44.215959 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb16172-2fb8-4cb1-a5d7-743ea26859c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:44.216236 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb16172-2fb8-4cb1-a5d7-743ea26859c1 None admin] PATH_INFO: `/v3/domains/default/users/d9c7158eb3ce41ab8bb7a5d05da4a2f6/roles/163c30c3b3c246beb849ce176f02d9da` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:44.218153 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb16172-2fb8-4cb1-a5d7-743ea26859c1 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=d9c7158eb3ce41ab8bb7a5d05da4a2f6, role_id=163c30c3b3c246beb849ce176f02d9da)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:04:44.220336 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb16172-2fb8-4cb1-a5d7-743ea26859c1 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:04:44.230340 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de52f8c-ca99-4fe0-b548-8acfe5f10a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:04:44.230954 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de52f8c-ca99-4fe0-b548-8acfe5f10a12 None admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:44.231237 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de52f8c-ca99-4fe0-b548-8acfe5f10a12 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:44.231512 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de52f8c-ca99-4fe0-b548-8acfe5f10a12 None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:44.232219 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de52f8c-ca99-4fe0-b548-8acfe5f10a12 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:04:44.233595 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de52f8c-ca99-4fe0-b548-8acfe5f10a12 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:04:44.234065 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 22/43] 10.222.0.61 () {58 vars in 1361 bytes} [Mon Aug 22 19:04:43 2022] PUT /identity/v3/domains/default/users/d9c7158eb3ce41ab8bb7a5d05da4a2f6/roles/163c30c3b3c246beb849ce176f02d9da => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 19:04:44.234899 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb16172-2fb8-4cb1-a5d7-743ea26859c1 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:04:44.255703 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 22/44] 10.222.0.61 () {60 vars in 1131 bytes} [Mon Aug 22 19:04:44 2022] POST /identity/v3/roles => generated 223 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 19:04:44.356169 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb16172-2fb8-4cb1-a5d7-743ea26859c1 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:04:44.400625 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f97264-8895-4a06-bab9-3329065b8346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:04:44.401278 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f97264-8895-4a06-bab9-3329065b8346 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:44.401560 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f97264-8895-4a06-bab9-3329065b8346 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:44.401835 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f97264-8895-4a06-bab9-3329065b8346 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:44.403713 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f97264-8895-4a06-bab9-3329065b8346 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:04:44.405188 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f97264-8895-4a06-bab9-3329065b8346 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:04:44.407401 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 23/45] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:04:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:04:44.413234 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de52f8c-ca99-4fe0-b548-8acfe5f10a12 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:04:44.519892 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de52f8c-ca99-4fe0-b548-8acfe5f10a12 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:04:44.532394 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:04:44.554505 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfb1d48-f347-4b75-89a2-058764c2b869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:04:44.555121 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfb1d48-f347-4b75-89a2-058764c2b869 None admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:44.555405 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfb1d48-f347-4b75-89a2-058764c2b869 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:44.555719 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfb1d48-f347-4b75-89a2-058764c2b869 None admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:44.556381 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfb1d48-f347-4b75-89a2-058764c2b869 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:04:44.557818 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfb1d48-f347-4b75-89a2-058764c2b869 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:04:44.584474 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 23/46] 10.222.0.61 () {60 vars in 1141 bytes} [Mon Aug 22 19:04:44 2022] POST /identity/v3/projects => generated 312 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:04:46.145657 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fb7568-1b2e-4e56-ab0e-85ac1782c58a None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:46.146038 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fb7568-1b2e-4e56-ab0e-85ac1782c58a None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:46.146214 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fb7568-1b2e-4e56-ab0e-85ac1782c58a None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:46.147608 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 24/47] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:04:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:04:46.153135 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85460642-e53b-47b5-a5c7-1685745b8267 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:46.153575 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85460642-e53b-47b5-a5c7-1685745b8267 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:46.153824 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85460642-e53b-47b5-a5c7-1685745b8267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:46.171553 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-85460642-e53b-47b5-a5c7-1685745b8267 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:04:46.172784 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-85460642-e53b-47b5-a5c7-1685745b8267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:04:46.238443 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 24/48] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:04:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 19:04:46.464544 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4204411-2211-462f-8203-f3e547122b14 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:46.464833 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4204411-2211-462f-8203-f3e547122b14 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:46.465114 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4204411-2211-462f-8203-f3e547122b14 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:46.466459 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 25/49] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:04:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:04:46.471909 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0595aa2e-d4c4-49de-ba07-13908fc56038 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:46.472200 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0595aa2e-d4c4-49de-ba07-13908fc56038 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:46.472476 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0595aa2e-d4c4-49de-ba07-13908fc56038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:46.491140 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-0595aa2e-d4c4-49de-ba07-13908fc56038 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:04:46.492370 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-0595aa2e-d4c4-49de-ba07-13908fc56038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:04:46.504214 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4204411-2211-462f-8203-f3e547122b14 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:04:46.557983 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 25/50] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:04:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 19:04:46.563332 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:04:46.605355 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4204411-2211-462f-8203-f3e547122b14 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:04:46.638143 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8616a642-4802-4608-84ef-86bb4abac24d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:04:46.638767 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8616a642-4802-4608-84ef-86bb4abac24d None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:46.639046 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8616a642-4802-4608-84ef-86bb4abac24d None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:46.639316 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8616a642-4802-4608-84ef-86bb4abac24d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:46.649449 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8616a642-4802-4608-84ef-86bb4abac24d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 19:04:46.654216 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-8616a642-4802-4608-84ef-86bb4abac24d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 19:04:46.654216 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:04:46.654216 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:04:46.654216 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:04:46.654216 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:04:46.654216 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:04:46.654216 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:04:46.654216 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:04:46.654216 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:04:46.654216 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:04:46.654216 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:04:46.654216 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:04:46.654216 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 19:04:46.654216 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 19:04:46.654216 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 19:04:46.654216 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 19:04:46.655307 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 19:04:46.655307 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 19:04:46.655307 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 19:04:46.655307 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 19:04:46.655307 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:04:46.655307 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:04:46.655307 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:04:46.655307 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:04:46.655307 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:04:46.655307 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:04:46.655307 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:04:46.655307 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:04:46.655307 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:04:46.655307 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:04:46.655307 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:04:46.655307 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:04:46.656284 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:04:46.656284 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:04:46.656284 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:04:46.656284 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:04:46.656284 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 19:04:46.656284 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 19:04:46.656284 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 19:04:46.656284 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 19:04:46.656284 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 19:04:46.656284 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 19:04:46.656284 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 19:04:46.656284 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:04:46.656284 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 26/51] 10.222.0.61 () {56 vars in 1096 bytes} [Mon Aug 22 19:04:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 19:04:46.662887 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0595aa2e-d4c4-49de-ba07-13908fc56038 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:04:46.688213 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:04:46.764833 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0595aa2e-d4c4-49de-ba07-13908fc56038 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:04:46.799006 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a27c7a5-76be-4816-8863-8cf52e3c4c2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:04:46.799613 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a27c7a5-76be-4816-8863-8cf52e3c4c2c None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:46.799916 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a27c7a5-76be-4816-8863-8cf52e3c4c2c None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:46.800189 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a27c7a5-76be-4816-8863-8cf52e3c4c2c None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:46.800857 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a27c7a5-76be-4816-8863-8cf52e3c4c2c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:04:46.801157 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a27c7a5-76be-4816-8863-8cf52e3c4c2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:04:46.802544 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a27c7a5-76be-4816-8863-8cf52e3c4c2c None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:04:46.810792 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 26/52] 10.222.0.61 () {56 vars in 1099 bytes} [Mon Aug 22 19:04:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:04:46.816242 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8616a642-4802-4608-84ef-86bb4abac24d None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:04:46.824511 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a27c7a5-76be-4816-8863-8cf52e3c4c2c None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:04:46.908069 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:04:46.916281 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8616a642-4802-4608-84ef-86bb4abac24d None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:04:46.923212 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a27c7a5-76be-4816-8863-8cf52e3c4c2c None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:04:46.933991 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:04:46.950902 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52594f97-a2a1-44b7-98be-a2700d3834e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:04:46.951535 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52594f97-a2a1-44b7-98be-a2700d3834e7 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:46.951846 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52594f97-a2a1-44b7-98be-a2700d3834e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:46.952124 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52594f97-a2a1-44b7-98be-a2700d3834e7 None admin] PATH_INFO: `/v3/users/d9c7158eb3ce41ab8bb7a5d05da4a2f6` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:46.953284 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52594f97-a2a1-44b7-98be-a2700d3834e7 None admin] RBAC: Authorizing `identity:get_user(user_id=d9c7158eb3ce41ab8bb7a5d05da4a2f6)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:04:46.954730 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52594f97-a2a1-44b7-98be-a2700d3834e7 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:04:46.956375 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 27/53] 10.222.0.61 () {56 vars in 1177 bytes} [Mon Aug 22 19:04:46 2022] GET /identity/v3/users/d9c7158eb3ce41ab8bb7a5d05da4a2f6 => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:04:46.956528 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67821957-88cc-459d-a52e-fcd523403683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:04:46.957162 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67821957-88cc-459d-a52e-fcd523403683 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:46.957444 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67821957-88cc-459d-a52e-fcd523403683 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:46.957716 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67821957-88cc-459d-a52e-fcd523403683 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:46.958852 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67821957-88cc-459d-a52e-fcd523403683 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:04:46.960326 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67821957-88cc-459d-a52e-fcd523403683 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:04:46.961807 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52594f97-a2a1-44b7-98be-a2700d3834e7 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:04:46.961927 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 27/54] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:04:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 19:04:46.966986 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67821957-88cc-459d-a52e-fcd523403683 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:04:47.066388 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67821957-88cc-459d-a52e-fcd523403683 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:04:47.069595 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52594f97-a2a1-44b7-98be-a2700d3834e7 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:04:47.099739 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50285cf1-96ed-447d-b6aa-0513c3208545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:04:47.100344 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50285cf1-96ed-447d-b6aa-0513c3208545 None admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:47.100623 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50285cf1-96ed-447d-b6aa-0513c3208545 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:47.100894 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50285cf1-96ed-447d-b6aa-0513c3208545 None admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:47.101532 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50285cf1-96ed-447d-b6aa-0513c3208545 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:04:47.103329 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50285cf1-96ed-447d-b6aa-0513c3208545 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:04:47.103704 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb160005-193c-47f7-b380-9e525bbb42cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:04:47.104343 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb160005-193c-47f7-b380-9e525bbb42cf None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:47.104625 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb160005-193c-47f7-b380-9e525bbb42cf None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:47.104900 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb160005-193c-47f7-b380-9e525bbb42cf None admin] PATH_INFO: `/v3/domains/default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:47.106019 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb160005-193c-47f7-b380-9e525bbb42cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:04:47.107489 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb160005-193c-47f7-b380-9e525bbb42cf None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:04:47.114169 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 28/55] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:04:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 19:04:47.115023 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb160005-193c-47f7-b380-9e525bbb42cf None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:04:47.130721 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 28/56] 10.222.0.61 () {60 vars in 1140 bytes} [Mon Aug 22 19:04:46 2022] POST /identity/v3/projects => generated 298 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 19:04:47.178529 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:04:47.226276 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb160005-193c-47f7-b380-9e525bbb42cf None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:04:47.260987 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9beb13-3a95-4ed5-b2cb-665f8fb35863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:04:47.261596 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9beb13-3a95-4ed5-b2cb-665f8fb35863 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:47.261869 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9beb13-3a95-4ed5-b2cb-665f8fb35863 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:47.262143 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9beb13-3a95-4ed5-b2cb-665f8fb35863 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:47.262883 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9beb13-3a95-4ed5-b2cb-665f8fb35863 None admin] RBAC: Adding query filter params (role.id=163c30c3b3c246beb849ce176f02d9da, scope.domain.id=default, user.id=d9c7158eb3ce41ab8bb7a5d05da4a2f6) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:04:47.263186 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9beb13-3a95-4ed5-b2cb-665f8fb35863 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:04:47.264650 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9beb13-3a95-4ed5-b2cb-665f8fb35863 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:04:47.271353 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 29/57] 10.222.0.61 () {56 vars in 1322 bytes} [Mon Aug 22 19:04:47 2022] GET /identity/v3/role_assignments?user.id=d9c7158eb3ce41ab8bb7a5d05da4a2f6&scope.domain.id=default&role.id=163c30c3b3c246beb849ce176f02d9da => generated 530 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 19:04:49.021319 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7688812-87d7-4647-a5ba-12e08604add3 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:49.021844 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7688812-87d7-4647-a5ba-12e08604add3 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:49.021927 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7688812-87d7-4647-a5ba-12e08604add3 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:49.023236 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 29/58] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:04:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:04:49.028449 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d80352-e28a-4fc3-ba3c-48bce4e75c58 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:49.028848 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d80352-e28a-4fc3-ba3c-48bce4e75c58 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:49.029017 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d80352-e28a-4fc3-ba3c-48bce4e75c58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:49.047788 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-26d80352-e28a-4fc3-ba3c-48bce4e75c58 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:04:49.049014 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-26d80352-e28a-4fc3-ba3c-48bce4e75c58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:04:49.115610 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 30/59] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:04:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 19:04:49.386711 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7688812-87d7-4647-a5ba-12e08604add3 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:04:49.396043 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:04:49.487773 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7688812-87d7-4647-a5ba-12e08604add3 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:04:49.522190 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca5145b-3d0c-4870-96ef-ecc8724cc0ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:04:49.522814 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca5145b-3d0c-4870-96ef-ecc8724cc0ed None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:49.523094 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca5145b-3d0c-4870-96ef-ecc8724cc0ed None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:49.523366 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca5145b-3d0c-4870-96ef-ecc8724cc0ed None admin] PATH_INFO: `/v3/domains/default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:49.524516 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca5145b-3d0c-4870-96ef-ecc8724cc0ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:04:49.525954 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca5145b-3d0c-4870-96ef-ecc8724cc0ed None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:04:49.527504 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 30/60] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:04:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:04:49.533564 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d80352-e28a-4fc3-ba3c-48bce4e75c58 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:04:49.580686 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:04:49.630056 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d80352-e28a-4fc3-ba3c-48bce4e75c58 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:04:49.663595 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0719d861-b414-4f68-b4d4-4193f42c94a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:04:49.664165 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0719d861-b414-4f68-b4d4-4193f42c94a7 None admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:49.664445 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0719d861-b414-4f68-b4d4-4193f42c94a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:49.664717 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0719d861-b414-4f68-b4d4-4193f42c94a7 None admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:49.665369 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0719d861-b414-4f68-b4d4-4193f42c94a7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:04:49.666803 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0719d861-b414-4f68-b4d4-4193f42c94a7 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:04:49.692275 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 31/61] 10.222.0.61 () {60 vars in 1132 bytes} [Mon Aug 22 19:04:49 2022] POST /identity/v3/users => generated 275 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:04:52.585495 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3d24eb-38cd-48fe-ba01-4ef8ddf3aff3 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:52.586012 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3d24eb-38cd-48fe-ba01-4ef8ddf3aff3 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:52.586124 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3d24eb-38cd-48fe-ba01-4ef8ddf3aff3 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:52.587396 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 31/62] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:04:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:04:52.613879 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e486b0a3-1968-4073-af19-ed3827677188 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:52.614377 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e486b0a3-1968-4073-af19-ed3827677188 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:52.614450 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e486b0a3-1968-4073-af19-ed3827677188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:52.641173 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-e486b0a3-1968-4073-af19-ed3827677188 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:04:52.642405 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-e486b0a3-1968-4073-af19-ed3827677188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:04:52.738477 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 32/63] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:04:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 19:04:52.810058 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5741a4aa-f5ae-49bf-b062-d7ddaa4352bd None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:52.810404 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5741a4aa-f5ae-49bf-b062-d7ddaa4352bd None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:52.810620 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5741a4aa-f5ae-49bf-b062-d7ddaa4352bd None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:52.812168 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 32/64] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:04:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:04:52.828802 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75229f6b-7ec6-4c17-a26a-f03033a04283 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:52.829099 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75229f6b-7ec6-4c17-a26a-f03033a04283 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:52.829382 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75229f6b-7ec6-4c17-a26a-f03033a04283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:52.853659 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-75229f6b-7ec6-4c17-a26a-f03033a04283 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:04:52.854789 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-75229f6b-7ec6-4c17-a26a-f03033a04283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:04:52.887523 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:04:52.951080 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 33/65] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:04:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 19:04:53.002945 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bab18a2-acb5-427b-864e-c07bd7895e73 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:53.003594 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bab18a2-acb5-427b-864e-c07bd7895e73 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:53.004117 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bab18a2-acb5-427b-864e-c07bd7895e73 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:53.005669 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 33/66] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:04:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:04:53.010921 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7be3b8-7655-47da-a6b2-769df4704a82 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:53.011211 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7be3b8-7655-47da-a6b2-769df4704a82 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:53.011483 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7be3b8-7655-47da-a6b2-769df4704a82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:53.032712 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-be7be3b8-7655-47da-a6b2-769df4704a82 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:04:53.033952 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-be7be3b8-7655-47da-a6b2-769df4704a82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:04:53.140338 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 34/67] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:04:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 19:04:53.161060 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bab18a2-acb5-427b-864e-c07bd7895e73 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:04:53.290449 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7be3b8-7655-47da-a6b2-769df4704a82 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:04:53.292472 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:04:53.326768 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bab18a2-acb5-427b-864e-c07bd7895e73 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:04:53.364148 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fe659b-1bb1-4650-8cc2-2afc676dc81b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:04:53.364793 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fe659b-1bb1-4650-8cc2-2afc676dc81b None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:53.365079 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fe659b-1bb1-4650-8cc2-2afc676dc81b None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:53.365364 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fe659b-1bb1-4650-8cc2-2afc676dc81b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:53.375894 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fe659b-1bb1-4650-8cc2-2afc676dc81b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 19:04:53.380786 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-04fe659b-1bb1-4650-8cc2-2afc676dc81b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 22 19:04:53.380786 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:04:53.380786 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:04:53.380786 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:04:53.380786 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:04:53.380786 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:04:53.380786 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:04:53.380786 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:04:53.380786 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:04:53.380786 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:04:53.380786 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:04:53.380786 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:04:53.380786 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 19:04:53.380786 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 19:04:53.380786 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 19:04:53.380786 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 19:04:53.381711 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 19:04:53.381711 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 19:04:53.381711 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 19:04:53.381711 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 19:04:53.381711 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:04:53.381711 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:04:53.381711 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:04:53.381711 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:04:53.381711 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:04:53.381711 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:04:53.381711 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:04:53.381711 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:04:53.381711 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:04:53.381711 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:04:53.381711 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:04:53.381711 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:04:53.382507 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:04:53.382507 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:04:53.382507 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:04:53.382507 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:04:53.382507 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 19:04:53.382507 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 19:04:53.382507 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 19:04:53.382507 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 19:04:53.382507 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 19:04:53.382507 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 19:04:53.382507 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 22 19:04:53.382507 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:04:53.382507 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 34/68] 10.222.0.61 () {56 vars in 1099 bytes} [Mon Aug 22 19:04:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:04:53.383485 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8343db0-b713-48a4-8120-960488dc016a None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:53.383948 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8343db0-b713-48a4-8120-960488dc016a None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:53.384391 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8343db0-b713-48a4-8120-960488dc016a None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:53.386065 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 35/69] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:04:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:04:53.387352 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4528d6f0-bd9a-4156-a452-eea3f40c4365 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:53.387793 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4528d6f0-bd9a-4156-a452-eea3f40c4365 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:53.388198 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4528d6f0-bd9a-4156-a452-eea3f40c4365 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:53.389544 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 36/70] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:04:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:04:53.394977 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4528d6f0-bd9a-4156-a452-eea3f40c4365 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:04:53.412390 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:04:53.421141 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7be3b8-7655-47da-a6b2-769df4704a82 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:04:53.456070 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979f7a2c-a719-434d-a22c-8617b83a2c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:04:53.456714 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979f7a2c-a719-434d-a22c-8617b83a2c7e None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:53.456995 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979f7a2c-a719-434d-a22c-8617b83a2c7e None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:53.457268 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979f7a2c-a719-434d-a22c-8617b83a2c7e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:53.467198 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979f7a2c-a719-434d-a22c-8617b83a2c7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 19:04:53.472576 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-979f7a2c-a719-434d-a22c-8617b83a2c7e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 19:04:53.472576 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:04:53.472576 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:04:53.472576 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:04:53.472576 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:04:53.472576 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:04:53.472576 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:04:53.472576 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:04:53.472576 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:04:53.472576 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:04:53.472576 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:04:53.472576 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:04:53.472576 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 19:04:53.472576 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 19:04:53.472576 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 19:04:53.472576 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 19:04:53.473470 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 19:04:53.473470 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 19:04:53.473470 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 19:04:53.473470 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 19:04:53.473470 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:04:53.473470 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:04:53.473470 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:04:53.473470 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:04:53.473470 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:04:53.473470 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:04:53.473470 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:04:53.473470 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:04:53.473470 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:04:53.473470 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:04:53.473470 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:04:53.473470 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:04:53.474294 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:04:53.474294 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:04:53.474294 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:04:53.474294 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:04:53.474294 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 19:04:53.474294 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 19:04:53.474294 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 19:04:53.474294 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 19:04:53.474294 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 19:04:53.474294 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 19:04:53.474294 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 19:04:53.474294 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:04:53.474294 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 35/71] 10.222.0.61 () {56 vars in 1096 bytes} [Mon Aug 22 19:04:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:04:53.475634 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de861cca-c1ca-4b66-ab80-fc3110065e50 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:53.475942 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de861cca-c1ca-4b66-ab80-fc3110065e50 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:53.476217 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de861cca-c1ca-4b66-ab80-fc3110065e50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:53.493315 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-de861cca-c1ca-4b66-ab80-fc3110065e50 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:04:53.494519 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-de861cca-c1ca-4b66-ab80-fc3110065e50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:04:53.502674 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4528d6f0-bd9a-4156-a452-eea3f40c4365 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:04:53.536474 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fc6ea3-6aaa-4644-8aa9-5f00a435d809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:04:53.537083 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fc6ea3-6aaa-4644-8aa9-5f00a435d809 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:53.537365 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fc6ea3-6aaa-4644-8aa9-5f00a435d809 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:53.537644 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fc6ea3-6aaa-4644-8aa9-5f00a435d809 None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:53.538313 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fc6ea3-6aaa-4644-8aa9-5f00a435d809 None admin] RBAC: Adding query filter params (name=member) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:04:53.538621 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fc6ea3-6aaa-4644-8aa9-5f00a435d809 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:04:53.540056 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fc6ea3-6aaa-4644-8aa9-5f00a435d809 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:04:53.541414 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:04:53.549196 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 37/72] 10.222.0.61 () {56 vars in 1101 bytes} [Mon Aug 22 19:04:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:04:53.550351 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1d28dc-4b08-4851-bb93-648d6d4294a5 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:53.550645 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1d28dc-4b08-4851-bb93-648d6d4294a5 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:53.550925 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1d28dc-4b08-4851-bb93-648d6d4294a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:53.561838 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 36/73] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:04:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 19:04:53.563381 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de861cca-c1ca-4b66-ab80-fc3110065e50 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:04:53.574797 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-7e1d28dc-4b08-4851-bb93-648d6d4294a5 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:04:53.576069 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-7e1d28dc-4b08-4851-bb93-648d6d4294a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:04:53.646133 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 38/74] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:04:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 19:04:53.646888 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1d28dc-4b08-4851-bb93-648d6d4294a5 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:04:53.670041 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:04:53.693062 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de861cca-c1ca-4b66-ab80-fc3110065e50 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:04:53.732454 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bfa6f7-4774-4798-a31f-ad8c5c36c04b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:04:53.733423 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bfa6f7-4774-4798-a31f-ad8c5c36c04b None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:53.733890 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bfa6f7-4774-4798-a31f-ad8c5c36c04b None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:53.734379 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bfa6f7-4774-4798-a31f-ad8c5c36c04b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:53.747901 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bfa6f7-4774-4798-a31f-ad8c5c36c04b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 19:04:53.753908 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-37bfa6f7-4774-4798-a31f-ad8c5c36c04b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 19:04:53.753908 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:04:53.753908 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:04:53.753908 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:04:53.753908 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:04:53.753908 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:04:53.753908 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:04:53.753908 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:04:53.753908 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:04:53.753908 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:04:53.753908 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:04:53.753908 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:04:53.753908 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 19:04:53.753908 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 19:04:53.753908 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 19:04:53.753908 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 19:04:53.754825 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 19:04:53.754825 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 19:04:53.754825 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 19:04:53.754825 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 19:04:53.754825 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:04:53.754825 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:04:53.754825 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:04:53.754825 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:04:53.754825 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:04:53.754825 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:04:53.754825 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:04:53.754825 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:04:53.754825 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:04:53.754825 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:04:53.754825 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:04:53.754825 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:04:53.755595 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:04:53.755595 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:04:53.755595 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:04:53.755595 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:04:53.755595 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 19:04:53.755595 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 19:04:53.755595 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 19:04:53.755595 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 19:04:53.755595 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 19:04:53.755595 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 19:04:53.755595 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 19:04:53.755595 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:04:53.758138 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 37/75] 10.222.0.61 () {56 vars in 1114 bytes} [Mon Aug 22 19:04:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:04:53.759193 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bfa6f7-4774-4798-a31f-ad8c5c36c04b None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:04:53.867010 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1d28dc-4b08-4851-bb93-648d6d4294a5 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:04:53.868206 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bfa6f7-4774-4798-a31f-ad8c5c36c04b None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:04:53.894539 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:04:53.904045 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ca44c6-713f-41aa-8a5b-83a6b7d68a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:04:53.904607 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ca44c6-713f-41aa-8a5b-83a6b7d68a48 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:53.904778 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799b7d1b-1d11-4345-ad39-698be659c948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:04:53.904885 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ca44c6-713f-41aa-8a5b-83a6b7d68a48 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:53.905152 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ca44c6-713f-41aa-8a5b-83a6b7d68a48 None admin] PATH_INFO: `/v3/users/a0683ebbf01f453fb5544a458adb02f4` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:53.905349 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799b7d1b-1d11-4345-ad39-698be659c948 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:53.905692 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799b7d1b-1d11-4345-ad39-698be659c948 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:53.905905 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799b7d1b-1d11-4345-ad39-698be659c948 None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:53.906611 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799b7d1b-1d11-4345-ad39-698be659c948 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:04:53.907041 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799b7d1b-1d11-4345-ad39-698be659c948 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:04:53.908597 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799b7d1b-1d11-4345-ad39-698be659c948 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:04:53.921319 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 39/76] 10.222.0.61 () {56 vars in 1099 bytes} [Mon Aug 22 19:04:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:04:53.921319 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799b7d1b-1d11-4345-ad39-698be659c948 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:04:53.924974 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ca44c6-713f-41aa-8a5b-83a6b7d68a48 None admin] RBAC: Authorizing `identity:get_user(user_id=a0683ebbf01f453fb5544a458adb02f4)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:04:53.926461 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ca44c6-713f-41aa-8a5b-83a6b7d68a48 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:04:53.928329 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 38/77] 10.222.0.61 () {56 vars in 1177 bytes} [Mon Aug 22 19:04:53 2022] GET /identity/v3/users/a0683ebbf01f453fb5544a458adb02f4 => generated 275 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:04:53.929024 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ca44c6-713f-41aa-8a5b-83a6b7d68a48 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:04:53.931786 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:04:54.025870 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799b7d1b-1d11-4345-ad39-698be659c948 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:04:54.027709 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ca44c6-713f-41aa-8a5b-83a6b7d68a48 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:04:54.058837 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79625c94-9bfd-42f1-8419-7358ed44689a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:04:54.059478 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79625c94-9bfd-42f1-8419-7358ed44689a None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:54.059798 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79625c94-9bfd-42f1-8419-7358ed44689a None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:54.060086 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79625c94-9bfd-42f1-8419-7358ed44689a None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:54.060751 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79625c94-9bfd-42f1-8419-7358ed44689a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:04:54.061056 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79625c94-9bfd-42f1-8419-7358ed44689a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:04:54.061305 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661f8dd7-0df7-457d-adb1-a5225ad7a53f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:04:54.061945 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661f8dd7-0df7-457d-adb1-a5225ad7a53f None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:54.062233 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661f8dd7-0df7-457d-adb1-a5225ad7a53f None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:54.063033 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79625c94-9bfd-42f1-8419-7358ed44689a None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:04:54.063136 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661f8dd7-0df7-457d-adb1-a5225ad7a53f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:54.064208 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661f8dd7-0df7-457d-adb1-a5225ad7a53f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:04:54.065630 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661f8dd7-0df7-457d-adb1-a5225ad7a53f None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:04:54.067205 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 39/78] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:04:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:04:54.069233 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661f8dd7-0df7-457d-adb1-a5225ad7a53f None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:04:54.079010 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 40/79] 10.222.0.61 () {56 vars in 1111 bytes} [Mon Aug 22 19:04:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:04:54.079010 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79625c94-9bfd-42f1-8419-7358ed44689a None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:04:54.128034 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:04:54.141274 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:04:54.175081 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661f8dd7-0df7-457d-adb1-a5225ad7a53f None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:04:54.178892 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79625c94-9bfd-42f1-8419-7358ed44689a None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:04:54.209354 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8e19e2-56fb-4f85-8643-2b64d0b1367e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:04:54.209988 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8e19e2-56fb-4f85-8643-2b64d0b1367e None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:54.210272 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8e19e2-56fb-4f85-8643-2b64d0b1367e None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:54.210549 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8e19e2-56fb-4f85-8643-2b64d0b1367e None admin] PATH_INFO: `/v3/users/d9c7158eb3ce41ab8bb7a5d05da4a2f6` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:54.211713 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8e19e2-56fb-4f85-8643-2b64d0b1367e None admin] RBAC: Authorizing `identity:get_user(user_id=d9c7158eb3ce41ab8bb7a5d05da4a2f6)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:04:54.212552 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac60d4e4-4ba4-4ab7-968d-bde40be0f635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:04:54.213160 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8e19e2-56fb-4f85-8643-2b64d0b1367e None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:04:54.213249 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac60d4e4-4ba4-4ab7-968d-bde40be0f635 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:54.213439 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac60d4e4-4ba4-4ab7-968d-bde40be0f635 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:54.213712 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac60d4e4-4ba4-4ab7-968d-bde40be0f635 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:54.214760 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 40/80] 10.222.0.61 () {56 vars in 1177 bytes} [Mon Aug 22 19:04:54 2022] GET /identity/v3/users/d9c7158eb3ce41ab8bb7a5d05da4a2f6 => generated 247 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:04:54.215526 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8e19e2-56fb-4f85-8643-2b64d0b1367e None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:04:54.282251 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac60d4e4-4ba4-4ab7-968d-bde40be0f635 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:04:54.283737 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac60d4e4-4ba4-4ab7-968d-bde40be0f635 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:04:54.317594 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8e19e2-56fb-4f85-8643-2b64d0b1367e None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:04:54.330799 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:04:54.352561 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c45f0e-6e8d-4007-ad13-01c3bb0fbf02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:04:54.353197 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c45f0e-6e8d-4007-ad13-01c3bb0fbf02 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:54.353477 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c45f0e-6e8d-4007-ad13-01c3bb0fbf02 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:54.353750 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c45f0e-6e8d-4007-ad13-01c3bb0fbf02 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:54.361002 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:04:54.366327 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c45f0e-6e8d-4007-ad13-01c3bb0fbf02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 19:04:54.371092 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-07c45f0e-6e8d-4007-ad13-01c3bb0fbf02 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 22 19:04:54.371092 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:04:54.371092 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:04:54.371092 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:04:54.371092 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:04:54.371092 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:04:54.371092 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:04:54.371092 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:04:54.371092 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:04:54.371092 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:04:54.371092 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:04:54.371092 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:04:54.371092 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 19:04:54.371092 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 19:04:54.371092 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 19:04:54.371092 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 19:04:54.372111 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 19:04:54.372111 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 19:04:54.372111 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 19:04:54.372111 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 19:04:54.372111 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:04:54.372111 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:04:54.372111 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:04:54.372111 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:04:54.372111 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:04:54.372111 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:04:54.372111 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:04:54.372111 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:04:54.372111 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:04:54.372111 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:04:54.372111 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:04:54.372111 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:04:54.372872 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:04:54.372872 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:04:54.372872 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:04:54.372872 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:04:54.372872 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 19:04:54.372872 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 19:04:54.372872 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 19:04:54.372872 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 19:04:54.372872 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 19:04:54.372872 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 19:04:54.372872 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 22 19:04:54.372872 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:04:54.372872 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 41/81] 10.222.0.61 () {56 vars in 1099 bytes} [Mon Aug 22 19:04:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:04:54.373542 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c45f0e-6e8d-4007-ad13-01c3bb0fbf02 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:04:54.389172 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 41/82] 10.222.0.61 () {58 vars in 1282 bytes} [Mon Aug 22 19:04:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 19:04:54.390009 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac60d4e4-4ba4-4ab7-968d-bde40be0f635 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:04:54.473187 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c45f0e-6e8d-4007-ad13-01c3bb0fbf02 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:04:54.487800 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac60d4e4-4ba4-4ab7-968d-bde40be0f635 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:04:54.507368 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f94a10-9267-409d-a24d-dfa437fe2675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:04:54.508018 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f94a10-9267-409d-a24d-dfa437fe2675 None admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:54.508292 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f94a10-9267-409d-a24d-dfa437fe2675 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:54.508564 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f94a10-9267-409d-a24d-dfa437fe2675 None admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:54.509208 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f94a10-9267-409d-a24d-dfa437fe2675 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:04:54.510672 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f94a10-9267-409d-a24d-dfa437fe2675 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:04:54.521249 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062e5a5f-0805-4d3d-8101-e2d082496550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:04:54.521911 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062e5a5f-0805-4d3d-8101-e2d082496550 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:54.522191 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062e5a5f-0805-4d3d-8101-e2d082496550 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:54.522469 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062e5a5f-0805-4d3d-8101-e2d082496550 None admin] PATH_INFO: `/v3/users/a0683ebbf01f453fb5544a458adb02f4` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:54.523748 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062e5a5f-0805-4d3d-8101-e2d082496550 None admin] RBAC: Authorizing `identity:get_user(user_id=a0683ebbf01f453fb5544a458adb02f4)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:04:54.525164 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062e5a5f-0805-4d3d-8101-e2d082496550 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:04:54.526748 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 42/83] 10.222.0.61 () {56 vars in 1177 bytes} [Mon Aug 22 19:04:54 2022] GET /identity/v3/users/a0683ebbf01f453fb5544a458adb02f4 => generated 275 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:04:54.527502 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062e5a5f-0805-4d3d-8101-e2d082496550 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:04:54.547674 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 42/84] 10.222.0.61 () {60 vars in 1132 bytes} [Mon Aug 22 19:04:54 2022] POST /identity/v3/users => generated 289 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:04:54.547674 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f94a10-9267-409d-a24d-dfa437fe2675 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:04:54.560212 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:04:54.603004 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:04:54.653349 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f94a10-9267-409d-a24d-dfa437fe2675 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:04:54.661596 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062e5a5f-0805-4d3d-8101-e2d082496550 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:04:54.693170 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb37af6-5b8c-4fe7-b8e5-cee9c615bbd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:04:54.693808 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb37af6-5b8c-4fe7-b8e5-cee9c615bbd4 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:54.694751 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb37af6-5b8c-4fe7-b8e5-cee9c615bbd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:54.694751 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb37af6-5b8c-4fe7-b8e5-cee9c615bbd4 None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:54.695744 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb37af6-5b8c-4fe7-b8e5-cee9c615bbd4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:04:54.695744 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb37af6-5b8c-4fe7-b8e5-cee9c615bbd4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:04:54.699046 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb37af6-5b8c-4fe7-b8e5-cee9c615bbd4 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:04:54.703870 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f9396f-6d2a-4474-9418-0e5cf523b7d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:04:54.704530 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f9396f-6d2a-4474-9418-0e5cf523b7d9 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:54.704817 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f9396f-6d2a-4474-9418-0e5cf523b7d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:54.705456 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f9396f-6d2a-4474-9418-0e5cf523b7d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:54.708342 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 43/85] 10.222.0.61 () {56 vars in 1101 bytes} [Mon Aug 22 19:04:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:04:54.708699 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb37af6-5b8c-4fe7-b8e5-cee9c615bbd4 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:04:54.792895 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f9396f-6d2a-4474-9418-0e5cf523b7d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:04:54.794584 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f9396f-6d2a-4474-9418-0e5cf523b7d9 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:04:54.818035 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:04:54.839072 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb37af6-5b8c-4fe7-b8e5-cee9c615bbd4 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:04:54.862798 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:04:54.884499 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f59f57-d564-4aec-8242-d57f2b6182e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:04:54.885145 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f59f57-d564-4aec-8242-d57f2b6182e7 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:54.885431 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f59f57-d564-4aec-8242-d57f2b6182e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:54.885711 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f59f57-d564-4aec-8242-d57f2b6182e7 None admin] PATH_INFO: `/v3/projects/a4fe339e17324d15a22e51626818eecd` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:54.886869 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f59f57-d564-4aec-8242-d57f2b6182e7 None admin] RBAC: Authorizing `identity:get_project(project_id=a4fe339e17324d15a22e51626818eecd)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:04:54.889390 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f59f57-d564-4aec-8242-d57f2b6182e7 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:04:54.890332 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 44/86] 10.222.0.61 () {56 vars in 1186 bytes} [Mon Aug 22 19:04:54 2022] GET /identity/v3/projects/a4fe339e17324d15a22e51626818eecd => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:04:54.891362 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f59f57-d564-4aec-8242-d57f2b6182e7 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:04:54.929803 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 43/87] 10.222.0.61 () {58 vars in 1282 bytes} [Mon Aug 22 19:04:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 19:04:54.930983 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f9396f-6d2a-4474-9418-0e5cf523b7d9 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:04:55.004922 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f59f57-d564-4aec-8242-d57f2b6182e7 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:04:55.043033 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691e8d6b-c345-407b-bd5d-4036d5c41fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:04:55.043342 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f9396f-6d2a-4474-9418-0e5cf523b7d9 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:04:55.044168 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691e8d6b-c345-407b-bd5d-4036d5c41fac None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:55.044684 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691e8d6b-c345-407b-bd5d-4036d5c41fac None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:55.045216 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691e8d6b-c345-407b-bd5d-4036d5c41fac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:55.061953 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:04:55.081863 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bb8393-090a-419b-9832-f5e147c137ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:04:55.082674 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bb8393-090a-419b-9832-f5e147c137ce None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:55.082982 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bb8393-090a-419b-9832-f5e147c137ce None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:55.083247 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bb8393-090a-419b-9832-f5e147c137ce None admin] PATH_INFO: `/v3/users/a0683ebbf01f453fb5544a458adb02f4` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:55.084491 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bb8393-090a-419b-9832-f5e147c137ce None admin] RBAC: Authorizing `identity:get_user(user_id=a0683ebbf01f453fb5544a458adb02f4)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:04:55.086026 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bb8393-090a-419b-9832-f5e147c137ce None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:04:55.087989 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 44/88] 10.222.0.61 () {56 vars in 1177 bytes} [Mon Aug 22 19:04:54 2022] GET /identity/v3/users/a0683ebbf01f453fb5544a458adb02f4 => generated 275 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:04:55.093239 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bb8393-090a-419b-9832-f5e147c137ce None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:04:55.106927 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:04:55.126613 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691e8d6b-c345-407b-bd5d-4036d5c41fac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:04:55.128494 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691e8d6b-c345-407b-bd5d-4036d5c41fac None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:04:55.201444 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bb8393-090a-419b-9832-f5e147c137ce None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:04:55.237888 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715a1298-f76c-414b-acb7-791da19a1b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:04:55.238864 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715a1298-f76c-414b-acb7-791da19a1b88 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:55.239210 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715a1298-f76c-414b-acb7-791da19a1b88 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:55.239551 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715a1298-f76c-414b-acb7-791da19a1b88 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:55.239671 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 45/89] 10.222.0.61 () {58 vars in 1282 bytes} [Mon Aug 22 19:04:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 19:04:55.240581 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691e8d6b-c345-407b-bd5d-4036d5c41fac None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:04:55.244844 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715a1298-f76c-414b-acb7-791da19a1b88 None admin] RBAC: Adding query filter params (role.id=74a40bc819cb42eea7e966a4b2a016f1, scope.project.id=a4fe339e17324d15a22e51626818eecd, user.id=a0683ebbf01f453fb5544a458adb02f4) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:04:55.245286 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715a1298-f76c-414b-acb7-791da19a1b88 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:04:55.246805 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715a1298-f76c-414b-acb7-791da19a1b88 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:04:55.256687 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 45/90] 10.222.0.61 () {56 vars in 1374 bytes} [Mon Aug 22 19:04:55 2022] GET /identity/v3/role_assignments?user.id=a0683ebbf01f453fb5544a458adb02f4&scope.project.id=a4fe339e17324d15a22e51626818eecd&role.id=74a40bc819cb42eea7e966a4b2a016f1 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:04:55.257441 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715a1298-f76c-414b-acb7-791da19a1b88 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:04:55.313592 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:04:55.368918 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:04:55.387055 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691e8d6b-c345-407b-bd5d-4036d5c41fac None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:04:55.410864 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715a1298-f76c-414b-acb7-791da19a1b88 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:04:55.423924 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a60d45-90dd-4275-b39f-d429498b2e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:04:55.424574 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a60d45-90dd-4275-b39f-d429498b2e1e None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:55.424854 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a60d45-90dd-4275-b39f-d429498b2e1e None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:55.425127 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a60d45-90dd-4275-b39f-d429498b2e1e None admin] PATH_INFO: `/v3/projects/a4fe339e17324d15a22e51626818eecd` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:55.426289 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a60d45-90dd-4275-b39f-d429498b2e1e None admin] RBAC: Authorizing `identity:get_project(project_id=a4fe339e17324d15a22e51626818eecd)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:04:55.427783 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a60d45-90dd-4275-b39f-d429498b2e1e None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:04:55.429602 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 46/91] 10.222.0.61 () {56 vars in 1186 bytes} [Mon Aug 22 19:04:55 2022] GET /identity/v3/projects/a4fe339e17324d15a22e51626818eecd => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:04:55.430280 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a60d45-90dd-4275-b39f-d429498b2e1e None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:04:55.483019 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14553c5c-e3f6-479f-9146-c20683e7c5c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:04:55.483795 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14553c5c-e3f6-479f-9146-c20683e7c5c2 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:55.484671 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14553c5c-e3f6-479f-9146-c20683e7c5c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:55.484964 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14553c5c-e3f6-479f-9146-c20683e7c5c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:55.539342 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a60d45-90dd-4275-b39f-d429498b2e1e None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:04:55.585126 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663a0416-7e80-4da9-b370-a50a4364ab45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:04:55.585696 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663a0416-7e80-4da9-b370-a50a4364ab45 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:55.585977 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663a0416-7e80-4da9-b370-a50a4364ab45 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:55.586249 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663a0416-7e80-4da9-b370-a50a4364ab45 None admin] PATH_INFO: `/v3/projects/a4fe339e17324d15a22e51626818eecd` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:55.588951 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663a0416-7e80-4da9-b370-a50a4364ab45 None admin] RBAC: Authorizing `identity:get_project(project_id=a4fe339e17324d15a22e51626818eecd)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:04:55.590414 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663a0416-7e80-4da9-b370-a50a4364ab45 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:04:55.592100 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 47/92] 10.222.0.61 () {56 vars in 1186 bytes} [Mon Aug 22 19:04:55 2022] GET /identity/v3/projects/a4fe339e17324d15a22e51626818eecd => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:04:55.592862 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663a0416-7e80-4da9-b370-a50a4364ab45 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:04:55.611429 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14553c5c-e3f6-479f-9146-c20683e7c5c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:04:55.613671 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14553c5c-e3f6-479f-9146-c20683e7c5c2 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:04:55.632041 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:04:55.673301 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:04:55.733811 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 46/93] 10.222.0.61 () {58 vars in 1282 bytes} [Mon Aug 22 19:04:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 19:04:55.735767 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14553c5c-e3f6-479f-9146-c20683e7c5c2 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:04:55.806914 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663a0416-7e80-4da9-b370-a50a4364ab45 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:04:55.845315 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fff242-e94f-49ba-94fd-103cd8abf747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:04:55.845856 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fff242-e94f-49ba-94fd-103cd8abf747 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:55.846138 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fff242-e94f-49ba-94fd-103cd8abf747 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:55.846416 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fff242-e94f-49ba-94fd-103cd8abf747 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:55.847160 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fff242-e94f-49ba-94fd-103cd8abf747 None admin] RBAC: Adding query filter params (role.id=163c30c3b3c246beb849ce176f02d9da, scope.project.id=a4fe339e17324d15a22e51626818eecd, user.id=d9c7158eb3ce41ab8bb7a5d05da4a2f6) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:04:55.847472 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fff242-e94f-49ba-94fd-103cd8abf747 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:04:55.849220 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fff242-e94f-49ba-94fd-103cd8abf747 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:04:55.855284 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 48/94] 10.222.0.61 () {56 vars in 1374 bytes} [Mon Aug 22 19:04:55 2022] GET /identity/v3/role_assignments?user.id=d9c7158eb3ce41ab8bb7a5d05da4a2f6&scope.project.id=a4fe339e17324d15a22e51626818eecd&role.id=163c30c3b3c246beb849ce176f02d9da => generated 259 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:04:55.856227 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fff242-e94f-49ba-94fd-103cd8abf747 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:04:55.949831 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14553c5c-e3f6-479f-9146-c20683e7c5c2 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:04:56.016060 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:04:56.037393 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3a79e7-449f-4ec6-94d4-11ed317626c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:04:56.038397 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3a79e7-449f-4ec6-94d4-11ed317626c2 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:56.038876 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3a79e7-449f-4ec6-94d4-11ed317626c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:56.039345 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3a79e7-449f-4ec6-94d4-11ed317626c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:56.040341 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3a79e7-449f-4ec6-94d4-11ed317626c2 None admin] RBAC: Adding query filter params (role.id=9f0952608a704dfca5fdbe4046331f21, scope.project.id=a4fe339e17324d15a22e51626818eecd, user.id=a0683ebbf01f453fb5544a458adb02f4) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:04:56.040906 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3a79e7-449f-4ec6-94d4-11ed317626c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:04:56.042578 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3a79e7-449f-4ec6-94d4-11ed317626c2 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:04:56.054858 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fff242-e94f-49ba-94fd-103cd8abf747 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:04:56.057119 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 47/95] 10.222.0.61 () {56 vars in 1374 bytes} [Mon Aug 22 19:04:55 2022] GET /identity/v3/role_assignments?user.id=a0683ebbf01f453fb5544a458adb02f4&scope.project.id=a4fe339e17324d15a22e51626818eecd&role.id=9f0952608a704dfca5fdbe4046331f21 => generated 259 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:04:56.121022 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a435dc-2e20-45de-8405-64b4ba3c3c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:04:56.121603 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a435dc-2e20-45de-8405-64b4ba3c3c71 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:56.121889 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a435dc-2e20-45de-8405-64b4ba3c3c71 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:56.122162 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a435dc-2e20-45de-8405-64b4ba3c3c71 None admin] PATH_INFO: `/v3/projects/f6d87df75a414a59b78d9ce3488c7059` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:56.123393 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a435dc-2e20-45de-8405-64b4ba3c3c71 None admin] RBAC: Authorizing `identity:get_project(project_id=f6d87df75a414a59b78d9ce3488c7059)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:04:56.125084 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a435dc-2e20-45de-8405-64b4ba3c3c71 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:04:56.127018 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 49/96] 10.222.0.61 () {56 vars in 1186 bytes} [Mon Aug 22 19:04:55 2022] GET /identity/v3/projects/f6d87df75a414a59b78d9ce3488c7059 => generated 312 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:04:56.132894 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3a79e7-449f-4ec6-94d4-11ed317626c2 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:04:56.173140 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:04:56.289941 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3a79e7-449f-4ec6-94d4-11ed317626c2 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:04:56.335891 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75506750-9c28-4d8c-8724-5ad03041dc4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:04:56.336901 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75506750-9c28-4d8c-8724-5ad03041dc4d None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:56.337378 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75506750-9c28-4d8c-8724-5ad03041dc4d None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:56.337879 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75506750-9c28-4d8c-8724-5ad03041dc4d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:56.338804 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75506750-9c28-4d8c-8724-5ad03041dc4d None admin] RBAC: Adding query filter params (role.id=74a40bc819cb42eea7e966a4b2a016f1, scope.project.id=f6d87df75a414a59b78d9ce3488c7059, user.id=a0683ebbf01f453fb5544a458adb02f4) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:04:56.339321 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75506750-9c28-4d8c-8724-5ad03041dc4d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:04:56.340992 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75506750-9c28-4d8c-8724-5ad03041dc4d None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:04:56.351991 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 48/97] 10.222.0.61 () {56 vars in 1374 bytes} [Mon Aug 22 19:04:56 2022] GET /identity/v3/role_assignments?user.id=a0683ebbf01f453fb5544a458adb02f4&scope.project.id=f6d87df75a414a59b78d9ce3488c7059&role.id=74a40bc819cb42eea7e966a4b2a016f1 => generated 259 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:04:57.095274 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c001c06c-b7e5-436d-a7b0-ed14af7a0fb0 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:57.095813 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c001c06c-b7e5-436d-a7b0-ed14af7a0fb0 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:57.095919 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c001c06c-b7e5-436d-a7b0-ed14af7a0fb0 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:57.097183 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 50/98] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:04:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:04:57.102383 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66adc95c-03cb-4546-add9-7164e0aa4353 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:57.102769 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66adc95c-03cb-4546-add9-7164e0aa4353 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:57.102941 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66adc95c-03cb-4546-add9-7164e0aa4353 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:57.126824 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c9995c-e432-4613-9607-28b0de675f40 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:57.127143 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c9995c-e432-4613-9607-28b0de675f40 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:57.127375 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c9995c-e432-4613-9607-28b0de675f40 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:57.128787 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 51/99] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:04:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 19:04:57.133871 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc62aaa-c0e0-4b12-9584-23ab6b6475c1 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:57.134156 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc62aaa-c0e0-4b12-9584-23ab6b6475c1 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:57.134430 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc62aaa-c0e0-4b12-9584-23ab6b6475c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:57.142659 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-66adc95c-03cb-4546-add9-7164e0aa4353 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:04:57.146800 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-66adc95c-03cb-4546-add9-7164e0aa4353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:04:57.173379 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-2bc62aaa-c0e0-4b12-9584-23ab6b6475c1 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:04:57.174969 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-2bc62aaa-c0e0-4b12-9584-23ab6b6475c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:04:57.320900 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 49/100] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:04:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 19:04:57.338915 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 52/101] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:04:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 19:04:57.773300 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66adc95c-03cb-4546-add9-7164e0aa4353 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:04:57.784440 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:04:57.874529 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc62aaa-c0e0-4b12-9584-23ab6b6475c1 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:04:57.911413 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66adc95c-03cb-4546-add9-7164e0aa4353 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:04:57.980908 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1e8a80-e000-4794-abf0-a3958fda3d7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:04:57.981913 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1e8a80-e000-4794-abf0-a3958fda3d7d None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:57.982389 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1e8a80-e000-4794-abf0-a3958fda3d7d None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:57.982880 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1e8a80-e000-4794-abf0-a3958fda3d7d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:58.001127 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1e8a80-e000-4794-abf0-a3958fda3d7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 19:04:58.006189 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-7b1e8a80-e000-4794-abf0-a3958fda3d7d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 22 19:04:58.006189 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:04:58.006189 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:04:58.006189 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:04:58.006189 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:04:58.006189 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:04:58.006189 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:04:58.006189 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:04:58.006189 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:04:58.006189 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:04:58.006189 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:04:58.006189 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:04:58.006189 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 19:04:58.006189 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 19:04:58.006189 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 19:04:58.006189 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 19:04:58.007032 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 19:04:58.007032 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 19:04:58.007032 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 19:04:58.007032 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 19:04:58.007032 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:04:58.007032 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:04:58.007032 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:04:58.007032 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:04:58.007032 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:04:58.007032 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:04:58.007032 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:04:58.007032 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:04:58.007032 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:04:58.007032 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:04:58.007032 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:04:58.007032 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:04:58.007860 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:04:58.007860 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:04:58.007860 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:04:58.007860 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:04:58.007860 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 19:04:58.007860 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 19:04:58.007860 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 19:04:58.007860 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 19:04:58.007860 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 19:04:58.007860 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 19:04:58.007860 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 22 19:04:58.007860 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:04:58.017714 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 50/102] 10.222.0.61 () {56 vars in 1099 bytes} [Mon Aug 22 19:04:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:04:58.024532 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1e8a80-e000-4794-abf0-a3958fda3d7d None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:04:58.027200 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:04:58.071039 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc62aaa-c0e0-4b12-9584-23ab6b6475c1 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:04:58.159137 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:04:58.163708 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2188a4-b0b5-41ad-90c2-c552affc9d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:04:58.164360 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2188a4-b0b5-41ad-90c2-c552affc9d57 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:58.164639 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2188a4-b0b5-41ad-90c2-c552affc9d57 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:58.164909 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2188a4-b0b5-41ad-90c2-c552affc9d57 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:58.170141 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2188a4-b0b5-41ad-90c2-c552affc9d57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:04:58.171597 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2188a4-b0b5-41ad-90c2-c552affc9d57 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:04:58.173560 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 53/103] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:04:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:04:58.174715 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf153e8-bbd9-4b6d-8b10-de3529227d0b None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:58.174997 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf153e8-bbd9-4b6d-8b10-de3529227d0b None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:58.175270 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf153e8-bbd9-4b6d-8b10-de3529227d0b None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:58.179031 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 54/104] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:04:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:04:58.179621 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf153e8-bbd9-4b6d-8b10-de3529227d0b None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:04:58.221765 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1e8a80-e000-4794-abf0-a3958fda3d7d None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:04:58.282385 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d5ebda-4f16-480b-8d06-04e678b45a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:04:58.282927 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d5ebda-4f16-480b-8d06-04e678b45a39 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:58.283212 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d5ebda-4f16-480b-8d06-04e678b45a39 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:58.283496 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d5ebda-4f16-480b-8d06-04e678b45a39 None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:58.284213 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d5ebda-4f16-480b-8d06-04e678b45a39 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:04:58.284765 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d5ebda-4f16-480b-8d06-04e678b45a39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:04:58.286749 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d5ebda-4f16-480b-8d06-04e678b45a39 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:04:58.305355 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 51/105] 10.222.0.61 () {56 vars in 1101 bytes} [Mon Aug 22 19:04:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:04:58.306612 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd3fc1b-bafe-4a00-ae01-0df1fcdc8bff None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:58.306898 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd3fc1b-bafe-4a00-ae01-0df1fcdc8bff None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:58.308498 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd3fc1b-bafe-4a00-ae01-0df1fcdc8bff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:58.341889 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-5dd3fc1b-bafe-4a00-ae01-0df1fcdc8bff None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:04:58.343110 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-5dd3fc1b-bafe-4a00-ae01-0df1fcdc8bff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:04:58.352855 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf153e8-bbd9-4b6d-8b10-de3529227d0b None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:04:58.410316 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d7dadd-36e9-4c66-8478-e3d7f0d2aa1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:04:58.411294 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d7dadd-36e9-4c66-8478-e3d7f0d2aa1f None admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:58.411805 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d7dadd-36e9-4c66-8478-e3d7f0d2aa1f None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:58.412306 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d7dadd-36e9-4c66-8478-e3d7f0d2aa1f None admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:58.413319 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d7dadd-36e9-4c66-8478-e3d7f0d2aa1f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:04:58.415536 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d7dadd-36e9-4c66-8478-e3d7f0d2aa1f None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:04:58.453183 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 55/106] 10.222.0.61 () {60 vars in 1141 bytes} [Mon Aug 22 19:04:58 2022] POST /identity/v3/projects => generated 302 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:04:58.460244 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d7dadd-36e9-4c66-8478-e3d7f0d2aa1f None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:04:58.463726 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:04:58.475231 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 52/107] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:04:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 19:04:58.477268 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c015a4c0-8e01-4341-904f-7ca5aebe100a None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:58.477821 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c015a4c0-8e01-4341-904f-7ca5aebe100a None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:58.483666 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c015a4c0-8e01-4341-904f-7ca5aebe100a None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:58.487738 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 53/108] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:04:58 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:04:58.502129 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440bdaaa-23e9-4f4f-ae3e-38401c53f784 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:58.502734 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440bdaaa-23e9-4f4f-ae3e-38401c53f784 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:58.503215 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440bdaaa-23e9-4f4f-ae3e-38401c53f784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:58.532861 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-440bdaaa-23e9-4f4f-ae3e-38401c53f784 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:04:58.541644 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-440bdaaa-23e9-4f4f-ae3e-38401c53f784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:04:58.653236 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:04:58.669967 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 54/109] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:04:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 19:04:58.679535 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d7dadd-36e9-4c66-8478-e3d7f0d2aa1f None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:04:58.721460 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbac7b19-41d0-475b-92a3-287e9f57b58d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:04:58.722112 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbac7b19-41d0-475b-92a3-287e9f57b58d None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:58.722401 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbac7b19-41d0-475b-92a3-287e9f57b58d None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:58.722677 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbac7b19-41d0-475b-92a3-287e9f57b58d None admin] PATH_INFO: `/v3/users/14147cb34fc743e89b373b1a740217f8` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:58.740804 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbac7b19-41d0-475b-92a3-287e9f57b58d None admin] RBAC: Authorizing `identity:get_user(user_id=14147cb34fc743e89b373b1a740217f8)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:04:58.742304 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbac7b19-41d0-475b-92a3-287e9f57b58d None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:04:58.744204 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 56/110] 10.222.0.61 () {56 vars in 1177 bytes} [Mon Aug 22 19:04:58 2022] GET /identity/v3/users/14147cb34fc743e89b373b1a740217f8 => generated 289 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:04:58.750388 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440bdaaa-23e9-4f4f-ae3e-38401c53f784 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:04:58.790777 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebdcfbd-443b-4f11-aacc-aadb28969f4d None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:58.791325 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebdcfbd-443b-4f11-aacc-aadb28969f4d None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:58.791615 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebdcfbd-443b-4f11-aacc-aadb28969f4d None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:58.793583 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 57/111] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:04:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:04:58.799771 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a403157-5e07-4af9-98be-00b644beafd2 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:58.800128 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a403157-5e07-4af9-98be-00b644beafd2 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:58.800389 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a403157-5e07-4af9-98be-00b644beafd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:58.831673 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-0a403157-5e07-4af9-98be-00b644beafd2 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:04:58.832886 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-0a403157-5e07-4af9-98be-00b644beafd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:04:58.915731 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440bdaaa-23e9-4f4f-ae3e-38401c53f784 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:04:58.921086 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:04:58.935336 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 58/112] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:04:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 19:04:58.936539 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a403157-5e07-4af9-98be-00b644beafd2 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:04:58.986682 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95256b7e-03ec-49e4-9a9a-b031fa741808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:04:58.989011 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95256b7e-03ec-49e4-9a9a-b031fa741808 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:58.989930 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95256b7e-03ec-49e4-9a9a-b031fa741808 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:58.990690 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95256b7e-03ec-49e4-9a9a-b031fa741808 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:59.065668 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a403157-5e07-4af9-98be-00b644beafd2 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:04:59.080064 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95256b7e-03ec-49e4-9a9a-b031fa741808 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:04:59.082353 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95256b7e-03ec-49e4-9a9a-b031fa741808 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:04:59.096036 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:04:59.107110 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d6bf10-583f-46bd-acdb-65a3c925eb8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:04:59.108035 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d6bf10-583f-46bd-acdb-65a3c925eb8b None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:59.108497 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d6bf10-583f-46bd-acdb-65a3c925eb8b None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:59.108950 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d6bf10-583f-46bd-acdb-65a3c925eb8b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:59.125318 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d6bf10-583f-46bd-acdb-65a3c925eb8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 19:04:59.131473 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-65d6bf10-583f-46bd-acdb-65a3c925eb8b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 22 19:04:59.131473 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:04:59.131473 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:04:59.131473 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:04:59.131473 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:04:59.131473 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:04:59.131473 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:04:59.131473 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:04:59.131473 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:04:59.131473 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:04:59.131473 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:04:59.131473 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:04:59.131473 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 19:04:59.131473 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 19:04:59.131473 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 19:04:59.131473 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 19:04:59.132369 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 19:04:59.132369 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 19:04:59.132369 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 19:04:59.132369 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 19:04:59.132369 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:04:59.132369 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:04:59.132369 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:04:59.132369 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:04:59.132369 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:04:59.132369 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:04:59.132369 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:04:59.132369 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:04:59.132369 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:04:59.132369 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:04:59.132369 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:04:59.132369 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:04:59.133151 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:04:59.133151 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:04:59.133151 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:04:59.133151 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:04:59.133151 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 19:04:59.133151 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 19:04:59.133151 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 19:04:59.133151 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 19:04:59.133151 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 19:04:59.133151 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 19:04:59.133151 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 22 19:04:59.133151 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:04:59.133151 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 59/113] 10.222.0.61 () {56 vars in 1099 bytes} [Mon Aug 22 19:04:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:04:59.133829 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d6bf10-583f-46bd-acdb-65a3c925eb8b None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:04:59.199305 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:04:59.211338 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 55/114] 10.222.0.61 () {58 vars in 1282 bytes} [Mon Aug 22 19:04:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 19:04:59.213252 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6a800a-967f-414e-a6a3-c4db318c7e95 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:59.213745 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6a800a-967f-414e-a6a3-c4db318c7e95 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:59.214224 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6a800a-967f-414e-a6a3-c4db318c7e95 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:59.215796 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 56/115] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:04:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:04:59.216704 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6a800a-967f-414e-a6a3-c4db318c7e95 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:04:59.255681 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d6bf10-583f-46bd-acdb-65a3c925eb8b None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:04:59.291709 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffcd1a0-9e4c-4a6c-bd3b-2736fb58cea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:04:59.292349 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffcd1a0-9e4c-4a6c-bd3b-2736fb58cea8 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:59.292631 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffcd1a0-9e4c-4a6c-bd3b-2736fb58cea8 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:59.292906 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffcd1a0-9e4c-4a6c-bd3b-2736fb58cea8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:59.304177 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffcd1a0-9e4c-4a6c-bd3b-2736fb58cea8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 19:04:59.309188 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-7ffcd1a0-9e4c-4a6c-bd3b-2736fb58cea8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 19:04:59.309188 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:04:59.309188 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:04:59.309188 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:04:59.309188 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:04:59.309188 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:04:59.309188 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:04:59.309188 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:04:59.309188 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:04:59.309188 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:04:59.309188 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:04:59.309188 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:04:59.309188 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 19:04:59.309188 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 19:04:59.309188 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 19:04:59.309188 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 19:04:59.310091 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 19:04:59.310091 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 19:04:59.310091 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 19:04:59.310091 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 19:04:59.310091 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:04:59.310091 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:04:59.310091 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:04:59.310091 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:04:59.310091 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:04:59.310091 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:04:59.310091 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:04:59.310091 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:04:59.310091 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:04:59.310091 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:04:59.310091 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:04:59.310091 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:04:59.310868 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:04:59.310868 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:04:59.310868 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:04:59.310868 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:04:59.310868 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 19:04:59.310868 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 19:04:59.310868 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 19:04:59.310868 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 19:04:59.310868 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 19:04:59.310868 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 19:04:59.310868 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 19:04:59.310868 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:04:59.310868 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 60/116] 10.222.0.61 () {56 vars in 1096 bytes} [Mon Aug 22 19:04:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:04:59.312129 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffcd1a0-9e4c-4a6c-bd3b-2736fb58cea8 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:04:59.319978 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:04:59.332578 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6a800a-967f-414e-a6a3-c4db318c7e95 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:04:59.373708 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7649ade5-6935-48ae-99d4-e0d5bdad77aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:04:59.375246 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7649ade5-6935-48ae-99d4-e0d5bdad77aa None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:59.375800 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7649ade5-6935-48ae-99d4-e0d5bdad77aa None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:59.376453 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7649ade5-6935-48ae-99d4-e0d5bdad77aa None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:59.377811 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7649ade5-6935-48ae-99d4-e0d5bdad77aa None admin] RBAC: Adding query filter params (name=member) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:04:59.378400 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7649ade5-6935-48ae-99d4-e0d5bdad77aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:04:59.380018 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7649ade5-6935-48ae-99d4-e0d5bdad77aa None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:04:59.394603 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 57/117] 10.222.0.61 () {56 vars in 1101 bytes} [Mon Aug 22 19:04:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:04:59.395995 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fe1eab-bf99-4f18-b780-ed6a22178527 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:59.396632 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fe1eab-bf99-4f18-b780-ed6a22178527 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:59.396989 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fe1eab-bf99-4f18-b780-ed6a22178527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:59.414840 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-d4fe1eab-bf99-4f18-b780-ed6a22178527 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:04:59.416150 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-d4fe1eab-bf99-4f18-b780-ed6a22178527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:04:59.421135 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffcd1a0-9e4c-4a6c-bd3b-2736fb58cea8 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:04:59.432616 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:04:59.460601 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944c1c4b-47fe-4b86-9d49-aca0534b1126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:04:59.461229 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944c1c4b-47fe-4b86-9d49-aca0534b1126 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:59.461624 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944c1c4b-47fe-4b86-9d49-aca0534b1126 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:59.461904 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944c1c4b-47fe-4b86-9d49-aca0534b1126 None admin] PATH_INFO: `/v3/projects/a4fe339e17324d15a22e51626818eecd` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:59.463176 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944c1c4b-47fe-4b86-9d49-aca0534b1126 None admin] RBAC: Authorizing `identity:get_project(project_id=a4fe339e17324d15a22e51626818eecd)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:04:59.464658 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944c1c4b-47fe-4b86-9d49-aca0534b1126 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:04:59.472002 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 61/118] 10.222.0.61 () {56 vars in 1186 bytes} [Mon Aug 22 19:04:59 2022] GET /identity/v3/projects/a4fe339e17324d15a22e51626818eecd => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:04:59.472002 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944c1c4b-47fe-4b86-9d49-aca0534b1126 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:04:59.491739 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 58/119] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:04:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 19:04:59.491739 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fe1eab-bf99-4f18-b780-ed6a22178527 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:04:59.586187 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:04:59.589243 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944c1c4b-47fe-4b86-9d49-aca0534b1126 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:04:59.608992 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fe1eab-bf99-4f18-b780-ed6a22178527 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:04:59.631325 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2dd5f4-e0f6-4ea8-98d4-04a5ce69ec7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:04:59.632357 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2dd5f4-e0f6-4ea8-98d4-04a5ce69ec7a None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:59.632716 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2dd5f4-e0f6-4ea8-98d4-04a5ce69ec7a None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:59.633066 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2dd5f4-e0f6-4ea8-98d4-04a5ce69ec7a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:59.646407 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc8c438-5fb1-4f52-9731-7105391a3738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:04:59.647298 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2dd5f4-e0f6-4ea8-98d4-04a5ce69ec7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 19:04:59.647413 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc8c438-5fb1-4f52-9731-7105391a3738 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:59.647836 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc8c438-5fb1-4f52-9731-7105391a3738 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:59.648261 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc8c438-5fb1-4f52-9731-7105391a3738 None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:59.649113 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc8c438-5fb1-4f52-9731-7105391a3738 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:04:59.649592 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc8c438-5fb1-4f52-9731-7105391a3738 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:04:59.651095 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc8c438-5fb1-4f52-9731-7105391a3738 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:04:59.652867 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-5b2dd5f4-e0f6-4ea8-98d4-04a5ce69ec7a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 19:04:59.652867 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:04:59.652867 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:04:59.652867 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:04:59.652867 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:04:59.652867 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:04:59.652867 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:04:59.652867 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:04:59.652867 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:04:59.652867 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:04:59.652867 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:04:59.652867 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:04:59.652867 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 19:04:59.652867 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 19:04:59.652867 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 19:04:59.652867 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 19:04:59.653897 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 19:04:59.653897 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 19:04:59.653897 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 19:04:59.653897 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 19:04:59.653897 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:04:59.653897 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:04:59.653897 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:04:59.653897 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:04:59.653897 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:04:59.653897 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:04:59.653897 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:04:59.653897 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:04:59.653897 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:04:59.653897 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:04:59.653897 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:04:59.653897 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:04:59.654863 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:04:59.654863 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:04:59.654863 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:04:59.654863 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:04:59.654863 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 19:04:59.654863 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 19:04:59.654863 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 19:04:59.654863 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 19:04:59.654863 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 19:04:59.654863 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 19:04:59.654863 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 19:04:59.654863 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:04:59.661032 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 62/120] 10.222.0.61 () {56 vars in 1114 bytes} [Mon Aug 22 19:04:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:04:59.661032 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2dd5f4-e0f6-4ea8-98d4-04a5ce69ec7a None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:04:59.668665 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 59/121] 10.222.0.61 () {56 vars in 1099 bytes} [Mon Aug 22 19:04:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:04:59.669844 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc8c438-5fb1-4f52-9731-7105391a3738 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:04:59.707910 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:04:59.775707 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2dd5f4-e0f6-4ea8-98d4-04a5ce69ec7a None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:04:59.804357 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc8c438-5fb1-4f52-9731-7105391a3738 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:04:59.813894 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1a6cfb-7840-4e62-a0a9-2384b587159e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:04:59.814546 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1a6cfb-7840-4e62-a0a9-2384b587159e None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:59.814828 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1a6cfb-7840-4e62-a0a9-2384b587159e None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:59.815105 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1a6cfb-7840-4e62-a0a9-2384b587159e None admin] PATH_INFO: `/v3/users/a0683ebbf01f453fb5544a458adb02f4` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:59.816259 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1a6cfb-7840-4e62-a0a9-2384b587159e None admin] RBAC: Authorizing `identity:get_user(user_id=a0683ebbf01f453fb5544a458adb02f4)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:04:59.817717 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1a6cfb-7840-4e62-a0a9-2384b587159e None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:04:59.819502 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 63/122] 10.222.0.61 () {56 vars in 1177 bytes} [Mon Aug 22 19:04:59 2022] GET /identity/v3/users/a0683ebbf01f453fb5544a458adb02f4 => generated 275 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:04:59.824559 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1a6cfb-7840-4e62-a0a9-2384b587159e None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:04:59.841128 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363abadd-4656-4e45-81fe-7a7ca9cf8d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:04:59.841514 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363abadd-4656-4e45-81fe-7a7ca9cf8d57 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:59.841791 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363abadd-4656-4e45-81fe-7a7ca9cf8d57 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:59.842062 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363abadd-4656-4e45-81fe-7a7ca9cf8d57 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:59.842824 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363abadd-4656-4e45-81fe-7a7ca9cf8d57 None admin] RBAC: Adding query filter params (role.id=3f7ffde3fddd46caa6e49c24a6f07673, scope.project.id=a4fe339e17324d15a22e51626818eecd, user.id=14147cb34fc743e89b373b1a740217f8) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:04:59.843134 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363abadd-4656-4e45-81fe-7a7ca9cf8d57 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:04:59.844830 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363abadd-4656-4e45-81fe-7a7ca9cf8d57 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:04:59.850868 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 60/123] 10.222.0.61 () {56 vars in 1374 bytes} [Mon Aug 22 19:04:59 2022] GET /identity/v3/role_assignments?user.id=14147cb34fc743e89b373b1a740217f8&scope.project.id=a4fe339e17324d15a22e51626818eecd&role.id=3f7ffde3fddd46caa6e49c24a6f07673 => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:04:59.851557 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363abadd-4656-4e45-81fe-7a7ca9cf8d57 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:04:59.871159 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:04:59.950053 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1a6cfb-7840-4e62-a0a9-2384b587159e None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:04:59.961314 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:04:59.995759 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363abadd-4656-4e45-81fe-7a7ca9cf8d57 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:04:59.997292 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d042209-5006-4f91-88d1-3da413e7b066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:04:59.997968 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d042209-5006-4f91-88d1-3da413e7b066 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:04:59.998343 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d042209-5006-4f91-88d1-3da413e7b066 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:04:59.998511 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d042209-5006-4f91-88d1-3da413e7b066 None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:04:59.999200 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d042209-5006-4f91-88d1-3da413e7b066 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:04:59.999518 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d042209-5006-4f91-88d1-3da413e7b066 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:00.001562 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d042209-5006-4f91-88d1-3da413e7b066 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:00.012836 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 64/124] 10.222.0.61 () {56 vars in 1111 bytes} [Mon Aug 22 19:04:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:00.018368 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d042209-5006-4f91-88d1-3da413e7b066 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:00.043709 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1a2625-42ca-4398-a25f-aa8f16262bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:00.044972 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1a2625-42ca-4398-a25f-aa8f16262bba None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:00.045331 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1a2625-42ca-4398-a25f-aa8f16262bba None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:00.045672 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1a2625-42ca-4398-a25f-aa8f16262bba None admin] PATH_INFO: `/v3/users/d9c7158eb3ce41ab8bb7a5d05da4a2f6` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:00.047731 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1a2625-42ca-4398-a25f-aa8f16262bba None admin] RBAC: Authorizing `identity:get_user(user_id=d9c7158eb3ce41ab8bb7a5d05da4a2f6)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:00.050127 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1a2625-42ca-4398-a25f-aa8f16262bba None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:00.052391 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 61/125] 10.222.0.61 () {56 vars in 1177 bytes} [Mon Aug 22 19:04:59 2022] GET /identity/v3/users/d9c7158eb3ce41ab8bb7a5d05da4a2f6 => generated 247 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:00.056229 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1a2625-42ca-4398-a25f-aa8f16262bba None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:00.145866 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d042209-5006-4f91-88d1-3da413e7b066 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:00.178211 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:00.190315 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1a2625-42ca-4398-a25f-aa8f16262bba None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:00.191856 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cad2fc5-2bb6-4957-9de1-27596bab9ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:00.192726 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cad2fc5-2bb6-4957-9de1-27596bab9ac2 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:00.193082 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cad2fc5-2bb6-4957-9de1-27596bab9ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:00.194210 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cad2fc5-2bb6-4957-9de1-27596bab9ac2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:00.207424 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cad2fc5-2bb6-4957-9de1-27596bab9ac2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 19:05:00.214011 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-5cad2fc5-2bb6-4957-9de1-27596bab9ac2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 22 19:05:00.214011 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:05:00.214011 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:05:00.214011 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:05:00.214011 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:05:00.214011 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:05:00.214011 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:05:00.214011 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:05:00.214011 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:05:00.214011 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:05:00.214011 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:05:00.214011 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:05:00.214011 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 19:05:00.214011 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 19:05:00.214011 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 19:05:00.214011 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 19:05:00.214874 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 19:05:00.214874 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 19:05:00.214874 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 19:05:00.214874 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 19:05:00.214874 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:05:00.214874 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:05:00.214874 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:05:00.214874 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:05:00.214874 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:05:00.214874 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:05:00.214874 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:05:00.214874 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:05:00.214874 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:05:00.214874 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:05:00.214874 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:05:00.214874 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:05:00.215887 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:05:00.215887 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:05:00.215887 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:05:00.215887 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:05:00.215887 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 19:05:00.215887 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 19:05:00.215887 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 19:05:00.215887 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 19:05:00.215887 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 19:05:00.215887 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 19:05:00.215887 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 22 19:05:00.215887 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:05:00.215887 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 65/126] 10.222.0.61 () {56 vars in 1099 bytes} [Mon Aug 22 19:05:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:05:00.216688 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cad2fc5-2bb6-4957-9de1-27596bab9ac2 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:00.236925 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfb061f-e5cb-4ffa-abee-93f66931c097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:00.237632 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfb061f-e5cb-4ffa-abee-93f66931c097 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:00.237917 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfb061f-e5cb-4ffa-abee-93f66931c097 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:00.238198 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfb061f-e5cb-4ffa-abee-93f66931c097 None admin] PATH_INFO: `/v3/projects/a4fe339e17324d15a22e51626818eecd` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:00.239800 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfb061f-e5cb-4ffa-abee-93f66931c097 None admin] RBAC: Authorizing `identity:get_project(project_id=a4fe339e17324d15a22e51626818eecd)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:00.241425 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfb061f-e5cb-4ffa-abee-93f66931c097 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:00.243292 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 62/127] 10.222.0.61 () {56 vars in 1186 bytes} [Mon Aug 22 19:05:00 2022] GET /identity/v3/projects/a4fe339e17324d15a22e51626818eecd => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:00.248635 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfb061f-e5cb-4ffa-abee-93f66931c097 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:00.250266 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:00.335914 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cad2fc5-2bb6-4957-9de1-27596bab9ac2 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:00.373469 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfb061f-e5cb-4ffa-abee-93f66931c097 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:00.382180 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9c54ab-664a-4213-8626-546a5d3063c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:00.383052 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9c54ab-664a-4213-8626-546a5d3063c6 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:00.383483 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9c54ab-664a-4213-8626-546a5d3063c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:00.384266 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9c54ab-664a-4213-8626-546a5d3063c6 None admin] PATH_INFO: `/v3/users/a0683ebbf01f453fb5544a458adb02f4` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:00.385895 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9c54ab-664a-4213-8626-546a5d3063c6 None admin] RBAC: Authorizing `identity:get_user(user_id=a0683ebbf01f453fb5544a458adb02f4)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:00.388016 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9c54ab-664a-4213-8626-546a5d3063c6 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:00.390582 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 66/128] 10.222.0.61 () {56 vars in 1177 bytes} [Mon Aug 22 19:05:00 2022] GET /identity/v3/users/a0683ebbf01f453fb5544a458adb02f4 => generated 275 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:00.391503 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9c54ab-664a-4213-8626-546a5d3063c6 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:00.411305 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642d43e4-761d-49dc-a328-ce96540c73ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:00.411925 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642d43e4-761d-49dc-a328-ce96540c73ab None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:00.412208 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642d43e4-761d-49dc-a328-ce96540c73ab None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:00.412530 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642d43e4-761d-49dc-a328-ce96540c73ab None admin] PATH_INFO: `/v3/projects/a4fe339e17324d15a22e51626818eecd` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:00.413686 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642d43e4-761d-49dc-a328-ce96540c73ab None admin] RBAC: Authorizing `identity:get_project(project_id=a4fe339e17324d15a22e51626818eecd)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:00.415223 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642d43e4-761d-49dc-a328-ce96540c73ab None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:00.421706 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 63/129] 10.222.0.61 () {56 vars in 1186 bytes} [Mon Aug 22 19:05:00 2022] GET /identity/v3/projects/a4fe339e17324d15a22e51626818eecd => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:00.423099 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642d43e4-761d-49dc-a328-ce96540c73ab None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:00.439191 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:00.504327 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:00.511436 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9c54ab-664a-4213-8626-546a5d3063c6 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:00.532789 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642d43e4-761d-49dc-a328-ce96540c73ab None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:00.549734 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f1252f-96e1-452e-9822-884c86af0305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:00.550568 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f1252f-96e1-452e-9822-884c86af0305 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:00.550998 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f1252f-96e1-452e-9822-884c86af0305 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:00.551419 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f1252f-96e1-452e-9822-884c86af0305 None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:00.552266 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f1252f-96e1-452e-9822-884c86af0305 None admin] RBAC: Adding query filter params (name=member) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:00.552705 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f1252f-96e1-452e-9822-884c86af0305 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:00.554254 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f1252f-96e1-452e-9822-884c86af0305 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:00.562873 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 67/130] 10.222.0.61 () {56 vars in 1101 bytes} [Mon Aug 22 19:05:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:00.563840 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f1252f-96e1-452e-9822-884c86af0305 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:00.574336 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ba1d37-5409-40ac-8e5d-ebf19f27c7e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:00.575017 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ba1d37-5409-40ac-8e5d-ebf19f27c7e2 None admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:00.575301 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ba1d37-5409-40ac-8e5d-ebf19f27c7e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:00.575578 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ba1d37-5409-40ac-8e5d-ebf19f27c7e2 None admin] PATH_INFO: `/v3/projects/a4fe339e17324d15a22e51626818eecd/users/a0683ebbf01f453fb5544a458adb02f4/roles/74a40bc819cb42eea7e966a4b2a016f1` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:00.577577 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ba1d37-5409-40ac-8e5d-ebf19f27c7e2 None admin] RBAC: Authorizing `identity:create_grant(project_id=a4fe339e17324d15a22e51626818eecd, user_id=a0683ebbf01f453fb5544a458adb02f4, role_id=74a40bc819cb42eea7e966a4b2a016f1)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:00.579677 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ba1d37-5409-40ac-8e5d-ebf19f27c7e2 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:00.596688 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 64/131] 10.222.0.61 () {58 vars in 1439 bytes} [Mon Aug 22 19:05:00 2022] PUT /identity/v3/projects/a4fe339e17324d15a22e51626818eecd/users/a0683ebbf01f453fb5544a458adb02f4/roles/74a40bc819cb42eea7e966a4b2a016f1 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:05:00.603674 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ba1d37-5409-40ac-8e5d-ebf19f27c7e2 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:00.683771 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:00.716436 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ba1d37-5409-40ac-8e5d-ebf19f27c7e2 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:00.742441 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f1252f-96e1-452e-9822-884c86af0305 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:00.754164 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df6d758-a5c9-44e5-8e94-4883e26a7713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:00.754938 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df6d758-a5c9-44e5-8e94-4883e26a7713 None admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:00.755225 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df6d758-a5c9-44e5-8e94-4883e26a7713 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:00.755515 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df6d758-a5c9-44e5-8e94-4883e26a7713 None admin] PATH_INFO: `/v3/projects/a4fe339e17324d15a22e51626818eecd/users/d9c7158eb3ce41ab8bb7a5d05da4a2f6/roles/163c30c3b3c246beb849ce176f02d9da` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:00.757606 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df6d758-a5c9-44e5-8e94-4883e26a7713 None admin] RBAC: Authorizing `identity:create_grant(project_id=a4fe339e17324d15a22e51626818eecd, user_id=d9c7158eb3ce41ab8bb7a5d05da4a2f6, role_id=163c30c3b3c246beb849ce176f02d9da)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:00.759184 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df6d758-a5c9-44e5-8e94-4883e26a7713 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:00.782469 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 65/132] 10.222.0.61 () {58 vars in 1439 bytes} [Mon Aug 22 19:05:00 2022] PUT /identity/v3/projects/a4fe339e17324d15a22e51626818eecd/users/d9c7158eb3ce41ab8bb7a5d05da4a2f6/roles/163c30c3b3c246beb849ce176f02d9da => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:05:00.783398 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df6d758-a5c9-44e5-8e94-4883e26a7713 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:00.824620 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08d561a-7db5-4bcf-a184-c10fea564647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:00.825293 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08d561a-7db5-4bcf-a184-c10fea564647 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:00.825574 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08d561a-7db5-4bcf-a184-c10fea564647 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:00.827255 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08d561a-7db5-4bcf-a184-c10fea564647 None admin] PATH_INFO: `/v3/projects/a4fe339e17324d15a22e51626818eecd` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:00.832310 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08d561a-7db5-4bcf-a184-c10fea564647 None admin] RBAC: Authorizing `identity:get_project(project_id=a4fe339e17324d15a22e51626818eecd)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:00.836384 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08d561a-7db5-4bcf-a184-c10fea564647 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:00.840344 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 68/133] 10.222.0.61 () {56 vars in 1186 bytes} [Mon Aug 22 19:05:00 2022] GET /identity/v3/projects/a4fe339e17324d15a22e51626818eecd => generated 298 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:00.847120 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7f9fa0-7442-476f-bab5-9711582de2c1 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:00.847435 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7f9fa0-7442-476f-bab5-9711582de2c1 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:00.848528 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7f9fa0-7442-476f-bab5-9711582de2c1 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:00.849481 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 69/134] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:05:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:05:00.850339 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7f9fa0-7442-476f-bab5-9711582de2c1 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:00.887303 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:00.966181 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df6d758-a5c9-44e5-8e94-4883e26a7713 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:01.002261 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7f9fa0-7442-476f-bab5-9711582de2c1 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:01.003026 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:01.009372 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c93bdb-25a2-4018-a7fa-b9ece525d860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:01.010081 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c93bdb-25a2-4018-a7fa-b9ece525d860 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:01.010366 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c93bdb-25a2-4018-a7fa-b9ece525d860 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:01.010646 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c93bdb-25a2-4018-a7fa-b9ece525d860 None admin] PATH_INFO: `/v3/users/a0683ebbf01f453fb5544a458adb02f4` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:01.011890 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c93bdb-25a2-4018-a7fa-b9ece525d860 None admin] RBAC: Authorizing `identity:get_user(user_id=a0683ebbf01f453fb5544a458adb02f4)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:01.013477 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c93bdb-25a2-4018-a7fa-b9ece525d860 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:01.015074 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 66/135] 10.222.0.61 () {56 vars in 1177 bytes} [Mon Aug 22 19:05:00 2022] GET /identity/v3/users/a0683ebbf01f453fb5544a458adb02f4 => generated 275 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:01.016481 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac814446-d147-40a8-aa49-e6286301accc None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:01.016771 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac814446-d147-40a8-aa49-e6286301accc None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:01.017051 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac814446-d147-40a8-aa49-e6286301accc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:01.036925 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-ac814446-d147-40a8-aa49-e6286301accc None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:05:01.038173 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-ac814446-d147-40a8-aa49-e6286301accc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:05:01.043473 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a379831d-3916-41e8-ae47-2e246807be34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:01.044196 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a379831d-3916-41e8-ae47-2e246807be34 None admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:01.044487 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a379831d-3916-41e8-ae47-2e246807be34 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:01.044759 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a379831d-3916-41e8-ae47-2e246807be34 None admin] PATH_INFO: `/v3/projects/a4fe339e17324d15a22e51626818eecd/users/a0683ebbf01f453fb5544a458adb02f4/roles/9f0952608a704dfca5fdbe4046331f21` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:01.048249 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a379831d-3916-41e8-ae47-2e246807be34 None admin] RBAC: Authorizing `identity:create_grant(project_id=a4fe339e17324d15a22e51626818eecd, user_id=a0683ebbf01f453fb5544a458adb02f4, role_id=9f0952608a704dfca5fdbe4046331f21)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:01.054067 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a379831d-3916-41e8-ae47-2e246807be34 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:01.068394 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 70/136] 10.222.0.61 () {58 vars in 1439 bytes} [Mon Aug 22 19:05:00 2022] PUT /identity/v3/projects/a4fe339e17324d15a22e51626818eecd/users/a0683ebbf01f453fb5544a458adb02f4/roles/9f0952608a704dfca5fdbe4046331f21 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:05:01.069398 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a379831d-3916-41e8-ae47-2e246807be34 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:01.180175 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 67/137] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:05:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 19:05:01.215967 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:01.235817 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a379831d-3916-41e8-ae47-2e246807be34 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:01.286856 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78eaa422-69ec-4e0b-bb5f-2e222ed552aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:01.287518 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eaa422-69ec-4e0b-bb5f-2e222ed552aa None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:01.287875 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eaa422-69ec-4e0b-bb5f-2e222ed552aa None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:01.288144 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eaa422-69ec-4e0b-bb5f-2e222ed552aa None admin] PATH_INFO: `/v3/projects/f6d87df75a414a59b78d9ce3488c7059` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:01.289369 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78eaa422-69ec-4e0b-bb5f-2e222ed552aa None admin] RBAC: Authorizing `identity:get_project(project_id=f6d87df75a414a59b78d9ce3488c7059)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:01.290863 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78eaa422-69ec-4e0b-bb5f-2e222ed552aa None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:01.292958 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 71/138] 10.222.0.61 () {56 vars in 1186 bytes} [Mon Aug 22 19:05:01 2022] GET /identity/v3/projects/f6d87df75a414a59b78d9ce3488c7059 => generated 312 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:01.299444 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac814446-d147-40a8-aa49-e6286301accc None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:01.414333 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac814446-d147-40a8-aa49-e6286301accc None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:01.453275 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:01.459286 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f7552d-1247-4eb1-97fc-24bc896cf3a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:01.460322 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f7552d-1247-4eb1-97fc-24bc896cf3a7 None admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:01.460962 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f7552d-1247-4eb1-97fc-24bc896cf3a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:01.461504 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f7552d-1247-4eb1-97fc-24bc896cf3a7 None admin] PATH_INFO: `/v3/projects/f6d87df75a414a59b78d9ce3488c7059/users/a0683ebbf01f453fb5544a458adb02f4/roles/74a40bc819cb42eea7e966a4b2a016f1` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:01.463953 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f7552d-1247-4eb1-97fc-24bc896cf3a7 None admin] RBAC: Authorizing `identity:create_grant(project_id=f6d87df75a414a59b78d9ce3488c7059, user_id=a0683ebbf01f453fb5544a458adb02f4, role_id=74a40bc819cb42eea7e966a4b2a016f1)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:01.465425 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f7552d-1247-4eb1-97fc-24bc896cf3a7 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:01.493439 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 68/139] 10.222.0.61 () {58 vars in 1439 bytes} [Mon Aug 22 19:05:01 2022] PUT /identity/v3/projects/f6d87df75a414a59b78d9ce3488c7059/users/a0683ebbf01f453fb5544a458adb02f4/roles/74a40bc819cb42eea7e966a4b2a016f1 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:05:01.598054 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78eaa422-69ec-4e0b-bb5f-2e222ed552aa None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:01.732527 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78eaa422-69ec-4e0b-bb5f-2e222ed552aa None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:01.779614 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb4d916-b3fe-48a5-b05c-a1404ea904e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:01.779944 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb4d916-b3fe-48a5-b05c-a1404ea904e6 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:01.780284 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb4d916-b3fe-48a5-b05c-a1404ea904e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:01.780581 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb4d916-b3fe-48a5-b05c-a1404ea904e6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:01.782808 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb4d916-b3fe-48a5-b05c-a1404ea904e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:01.784380 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb4d916-b3fe-48a5-b05c-a1404ea904e6 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:01.786309 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 72/140] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:05:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:01.791821 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f7552d-1247-4eb1-97fc-24bc896cf3a7 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:01.897978 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f7552d-1247-4eb1-97fc-24bc896cf3a7 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:01.934457 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a3a6c1-fadb-4341-9816-87156ead63dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:01.935132 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a3a6c1-fadb-4341-9816-87156ead63dd None admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:01.935418 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a3a6c1-fadb-4341-9816-87156ead63dd None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:01.935718 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a3a6c1-fadb-4341-9816-87156ead63dd None admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:01.936454 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a3a6c1-fadb-4341-9816-87156ead63dd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:01.937996 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a3a6c1-fadb-4341-9816-87156ead63dd None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:01.969754 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 69/141] 10.222.0.61 () {60 vars in 1132 bytes} [Mon Aug 22 19:05:01 2022] POST /identity/v3/users => generated 283 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:05:02.266176 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c11fd3-31f1-42d9-9bbc-62d9b09cf982 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:02.266690 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c11fd3-31f1-42d9-9bbc-62d9b09cf982 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:02.266762 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c11fd3-31f1-42d9-9bbc-62d9b09cf982 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:02.268423 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 73/142] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:05:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:05:02.281063 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cb156c-bfbf-47df-b41b-6e533de0bfae None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:02.281393 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cb156c-bfbf-47df-b41b-6e533de0bfae None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:02.281676 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cb156c-bfbf-47df-b41b-6e533de0bfae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:02.303878 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-69cb156c-bfbf-47df-b41b-6e533de0bfae None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:05:02.305085 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-69cb156c-bfbf-47df-b41b-6e533de0bfae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:05:02.349837 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:02.378445 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 70/143] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:05:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 19:05:02.678180 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c11fd3-31f1-42d9-9bbc-62d9b09cf982 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:02.705014 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:02.800251 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c11fd3-31f1-42d9-9bbc-62d9b09cf982 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:02.838658 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af72d6b-b082-4f37-890b-f879cf5f0578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:02.839635 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af72d6b-b082-4f37-890b-f879cf5f0578 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:02.840258 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af72d6b-b082-4f37-890b-f879cf5f0578 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:02.840731 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af72d6b-b082-4f37-890b-f879cf5f0578 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:02.854406 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af72d6b-b082-4f37-890b-f879cf5f0578 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 19:05:02.859582 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-9af72d6b-b082-4f37-890b-f879cf5f0578 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 22 19:05:02.859582 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:05:02.859582 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:05:02.859582 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:05:02.859582 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:05:02.859582 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:05:02.859582 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:05:02.859582 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:05:02.859582 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:05:02.859582 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:05:02.859582 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:05:02.859582 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:05:02.859582 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 19:05:02.859582 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 19:05:02.859582 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 19:05:02.859582 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 19:05:02.860457 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 19:05:02.860457 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 19:05:02.860457 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 19:05:02.860457 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 19:05:02.860457 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:05:02.860457 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:05:02.860457 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:05:02.860457 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:05:02.860457 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:05:02.860457 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:05:02.860457 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:05:02.860457 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:05:02.860457 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:05:02.860457 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:05:02.860457 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:05:02.860457 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:05:02.861230 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:05:02.861230 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:05:02.861230 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:05:02.861230 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:05:02.861230 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 19:05:02.861230 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 19:05:02.861230 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 19:05:02.861230 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 19:05:02.861230 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 19:05:02.861230 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 19:05:02.861230 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 22 19:05:02.861230 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:05:02.863810 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 74/144] 10.222.0.61 () {56 vars in 1099 bytes} [Mon Aug 22 19:05:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:05:02.870101 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cb156c-bfbf-47df-b41b-6e533de0bfae None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:02.990578 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cb156c-bfbf-47df-b41b-6e533de0bfae None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:03.032112 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8b7d27-5f3a-4147-9bf2-144bc6d0e73f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:03.033082 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8b7d27-5f3a-4147-9bf2-144bc6d0e73f None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:03.033583 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8b7d27-5f3a-4147-9bf2-144bc6d0e73f None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:03.034073 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8b7d27-5f3a-4147-9bf2-144bc6d0e73f None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:03.035085 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8b7d27-5f3a-4147-9bf2-144bc6d0e73f None admin] RBAC: Adding query filter params (name=reader) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:03.035589 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8b7d27-5f3a-4147-9bf2-144bc6d0e73f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:03.037197 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8b7d27-5f3a-4147-9bf2-144bc6d0e73f None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:03.047482 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 71/145] 10.222.0.61 () {56 vars in 1101 bytes} [Mon Aug 22 19:05:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:03.052791 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af72d6b-b082-4f37-890b-f879cf5f0578 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:03.153021 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:03.175573 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af72d6b-b082-4f37-890b-f879cf5f0578 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:03.218120 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7680cde-3d2e-4312-892b-c6b26ab35ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:03.219380 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7680cde-3d2e-4312-892b-c6b26ab35ae1 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:03.219685 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7680cde-3d2e-4312-892b-c6b26ab35ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:03.219966 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7680cde-3d2e-4312-892b-c6b26ab35ae1 None admin] PATH_INFO: `/v3/users/14147cb34fc743e89b373b1a740217f8` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:03.224372 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7680cde-3d2e-4312-892b-c6b26ab35ae1 None admin] RBAC: Authorizing `identity:get_user(user_id=14147cb34fc743e89b373b1a740217f8)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:03.225851 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7680cde-3d2e-4312-892b-c6b26ab35ae1 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:03.227930 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 75/146] 10.222.0.61 () {56 vars in 1177 bytes} [Mon Aug 22 19:05:03 2022] GET /identity/v3/users/14147cb34fc743e89b373b1a740217f8 => generated 289 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:03.233273 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8b7d27-5f3a-4147-9bf2-144bc6d0e73f None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:03.287235 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:03.361567 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8b7d27-5f3a-4147-9bf2-144bc6d0e73f None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:03.403182 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cb0e62-878c-4185-b2eb-787b70ec1a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:03.403850 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cb0e62-878c-4185-b2eb-787b70ec1a4b None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:03.404135 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cb0e62-878c-4185-b2eb-787b70ec1a4b None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:03.404413 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cb0e62-878c-4185-b2eb-787b70ec1a4b None admin] PATH_INFO: `/v3/projects/a4fe339e17324d15a22e51626818eecd` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:03.406388 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cb0e62-878c-4185-b2eb-787b70ec1a4b None admin] RBAC: Authorizing `identity:get_project(project_id=a4fe339e17324d15a22e51626818eecd)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:03.407940 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cb0e62-878c-4185-b2eb-787b70ec1a4b None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:03.409977 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 72/147] 10.222.0.61 () {56 vars in 1186 bytes} [Mon Aug 22 19:05:03 2022] GET /identity/v3/projects/a4fe339e17324d15a22e51626818eecd => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:03.415402 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7680cde-3d2e-4312-892b-c6b26ab35ae1 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:03.538951 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7680cde-3d2e-4312-892b-c6b26ab35ae1 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:03.577940 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f792bb-03eb-428f-9fac-ae17a2e6a788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:03.579500 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f792bb-03eb-428f-9fac-ae17a2e6a788 None admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:03.580022 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f792bb-03eb-428f-9fac-ae17a2e6a788 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:03.580496 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f792bb-03eb-428f-9fac-ae17a2e6a788 None admin] PATH_INFO: `/v3/projects/a4fe339e17324d15a22e51626818eecd/users/14147cb34fc743e89b373b1a740217f8/roles/3f7ffde3fddd46caa6e49c24a6f07673` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:03.582696 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f792bb-03eb-428f-9fac-ae17a2e6a788 None admin] RBAC: Authorizing `identity:create_grant(project_id=a4fe339e17324d15a22e51626818eecd, user_id=14147cb34fc743e89b373b1a740217f8, role_id=3f7ffde3fddd46caa6e49c24a6f07673)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:03.584381 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f792bb-03eb-428f-9fac-ae17a2e6a788 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:03.612636 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 76/148] 10.222.0.61 () {58 vars in 1439 bytes} [Mon Aug 22 19:05:03 2022] PUT /identity/v3/projects/a4fe339e17324d15a22e51626818eecd/users/14147cb34fc743e89b373b1a740217f8/roles/3f7ffde3fddd46caa6e49c24a6f07673 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:05:03.771531 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23a0b6a-196e-4201-95eb-b2a70d13ef75 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:03.771861 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23a0b6a-196e-4201-95eb-b2a70d13ef75 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:03.772131 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23a0b6a-196e-4201-95eb-b2a70d13ef75 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:03.773512 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 73/149] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:05:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:05:03.779116 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402aac34-0b4e-47d0-a0a7-d46e1ee502ed None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:03.779665 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402aac34-0b4e-47d0-a0a7-d46e1ee502ed None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:03.779943 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402aac34-0b4e-47d0-a0a7-d46e1ee502ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:03.804978 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-402aac34-0b4e-47d0-a0a7-d46e1ee502ed None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:05:03.806225 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-402aac34-0b4e-47d0-a0a7-d46e1ee502ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:05:03.809570 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:03.892677 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 77/150] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:05:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 19:05:04.112985 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b7144-e051-46af-a180-01401c24831e None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:04.113836 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b7144-e051-46af-a180-01401c24831e None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:04.114329 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b7144-e051-46af-a180-01401c24831e None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:04.115970 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 74/151] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:05:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:05:04.121999 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07f7902-85a1-49a2-b940-3788fa520c18 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:04.122290 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07f7902-85a1-49a2-b940-3788fa520c18 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:04.122562 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07f7902-85a1-49a2-b940-3788fa520c18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:04.156700 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-f07f7902-85a1-49a2-b940-3788fa520c18 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:05:04.157920 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-f07f7902-85a1-49a2-b940-3788fa520c18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:05:04.238270 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 78/152] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:05:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 19:05:04.305729 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a361a6-ca49-41d6-b57a-e82aac330c97 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:04.306366 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a361a6-ca49-41d6-b57a-e82aac330c97 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:04.306836 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a361a6-ca49-41d6-b57a-e82aac330c97 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:04.308459 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 75/153] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:05:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:05:04.313749 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf73ac7-e74e-4641-9a67-7c05bbe2e185 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:04.314037 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf73ac7-e74e-4641-9a67-7c05bbe2e185 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:04.314309 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf73ac7-e74e-4641-9a67-7c05bbe2e185 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:04.337165 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-1cf73ac7-e74e-4641-9a67-7c05bbe2e185 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:05:04.338378 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-1cf73ac7-e74e-4641-9a67-7c05bbe2e185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:05:04.405157 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a361a6-ca49-41d6-b57a-e82aac330c97 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:04.418461 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:04.426301 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 79/154] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:05:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 19:05:04.548613 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a361a6-ca49-41d6-b57a-e82aac330c97 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:04.557796 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:04.587389 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c3af5c-a630-4fb3-8355-b53b2c01ad1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:04.588061 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c3af5c-a630-4fb3-8355-b53b2c01ad1f None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:04.588394 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c3af5c-a630-4fb3-8355-b53b2c01ad1f None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:04.588666 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c3af5c-a630-4fb3-8355-b53b2c01ad1f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:04.599943 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c3af5c-a630-4fb3-8355-b53b2c01ad1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 19:05:04.605074 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-63c3af5c-a630-4fb3-8355-b53b2c01ad1f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 22 19:05:04.605074 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:05:04.605074 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:05:04.605074 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:05:04.605074 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:05:04.605074 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:05:04.605074 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:05:04.605074 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:05:04.605074 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:05:04.605074 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:05:04.605074 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:05:04.605074 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:05:04.605074 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 19:05:04.605074 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 19:05:04.605074 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 19:05:04.605074 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 19:05:04.605936 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 19:05:04.605936 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 19:05:04.605936 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 19:05:04.605936 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 19:05:04.605936 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:05:04.605936 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:05:04.605936 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:05:04.605936 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:05:04.605936 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:05:04.605936 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:05:04.605936 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:05:04.605936 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:05:04.605936 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:05:04.605936 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:05:04.605936 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:05:04.605936 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:05:04.606711 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:05:04.606711 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:05:04.606711 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:05:04.606711 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:05:04.606711 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 19:05:04.606711 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 19:05:04.606711 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 19:05:04.606711 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 19:05:04.606711 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 19:05:04.606711 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 19:05:04.606711 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 22 19:05:04.606711 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:05:04.606711 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 76/155] 10.222.0.61 () {56 vars in 1099 bytes} [Mon Aug 22 19:05:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 19:05:04.612365 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf73ac7-e74e-4641-9a67-7c05bbe2e185 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:04.723939 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf73ac7-e74e-4641-9a67-7c05bbe2e185 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:04.765265 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd79153-f5eb-493d-8df6-7265f7ce42b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:04.766545 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd79153-f5eb-493d-8df6-7265f7ce42b6 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:04.766934 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd79153-f5eb-493d-8df6-7265f7ce42b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:04.767211 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd79153-f5eb-493d-8df6-7265f7ce42b6 None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:04.771993 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd79153-f5eb-493d-8df6-7265f7ce42b6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:04.772319 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd79153-f5eb-493d-8df6-7265f7ce42b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:04.773764 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd79153-f5eb-493d-8df6-7265f7ce42b6 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:04.783452 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 80/156] 10.222.0.61 () {56 vars in 1101 bytes} [Mon Aug 22 19:05:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:04.789029 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c3af5c-a630-4fb3-8355-b53b2c01ad1f None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:04.889290 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd79153-f5eb-493d-8df6-7265f7ce42b6 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:04.945352 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c3af5c-a630-4fb3-8355-b53b2c01ad1f None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:05.001364 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ea6b6c-c61e-4eec-98e2-4ec6ae81736b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:05.001928 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ea6b6c-c61e-4eec-98e2-4ec6ae81736b None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:05.002207 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ea6b6c-c61e-4eec-98e2-4ec6ae81736b None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:05.002479 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ea6b6c-c61e-4eec-98e2-4ec6ae81736b None admin] PATH_INFO: `/v3/users/a0683ebbf01f453fb5544a458adb02f4` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:05.003635 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ea6b6c-c61e-4eec-98e2-4ec6ae81736b None admin] RBAC: Authorizing `identity:get_user(user_id=a0683ebbf01f453fb5544a458adb02f4)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:05.005420 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ea6b6c-c61e-4eec-98e2-4ec6ae81736b None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:05.007297 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 77/157] 10.222.0.61 () {56 vars in 1177 bytes} [Mon Aug 22 19:05:04 2022] GET /identity/v3/users/a0683ebbf01f453fb5544a458adb02f4 => generated 275 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:05.013296 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ea6b6c-c61e-4eec-98e2-4ec6ae81736b None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:05.032179 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:05.072023 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:05.103136 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd79153-f5eb-493d-8df6-7265f7ce42b6 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:05.148127 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ea6b6c-c61e-4eec-98e2-4ec6ae81736b None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:05.173459 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2aaece-1763-4ea5-9f97-6ee2dfa6170a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:05.174103 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2aaece-1763-4ea5-9f97-6ee2dfa6170a None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:05.174376 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2aaece-1763-4ea5-9f97-6ee2dfa6170a None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:05.174643 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2aaece-1763-4ea5-9f97-6ee2dfa6170a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:05.197102 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2aaece-1763-4ea5-9f97-6ee2dfa6170a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 19:05:05.201276 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e752469-f094-4f57-adbd-66eb581ddd7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:05.201963 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e752469-f094-4f57-adbd-66eb581ddd7d None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:05.202186 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e752469-f094-4f57-adbd-66eb581ddd7d None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:05.202466 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e752469-f094-4f57-adbd-66eb581ddd7d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:05.210566 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-7b2aaece-1763-4ea5-9f97-6ee2dfa6170a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 19:05:05.210566 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:05:05.210566 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:05:05.210566 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:05:05.210566 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:05:05.210566 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:05:05.210566 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:05:05.210566 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:05:05.210566 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:05:05.210566 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:05:05.210566 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:05:05.210566 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:05:05.210566 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 19:05:05.210566 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 19:05:05.210566 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 19:05:05.210566 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 19:05:05.211660 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 19:05:05.211660 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 19:05:05.211660 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 19:05:05.211660 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 19:05:05.211660 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:05:05.211660 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:05:05.211660 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:05:05.211660 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:05:05.211660 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:05:05.211660 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:05:05.211660 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:05:05.211660 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:05:05.211660 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:05:05.211660 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:05:05.211660 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:05:05.211660 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:05:05.212521 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:05:05.212521 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:05:05.212521 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:05:05.212521 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:05:05.212521 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 19:05:05.212521 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 19:05:05.212521 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 19:05:05.212521 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 19:05:05.212521 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 19:05:05.212521 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 19:05:05.212521 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 19:05:05.212521 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:05:05.212521 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 81/158] 10.222.0.61 () {56 vars in 1114 bytes} [Mon Aug 22 19:05:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 19:05:05.213159 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a581d1a1-ce66-4bc8-97d9-9a66e7d5bab8 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:05.213223 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a581d1a1-ce66-4bc8-97d9-9a66e7d5bab8 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:05.213817 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a581d1a1-ce66-4bc8-97d9-9a66e7d5bab8 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:05.214996 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 82/159] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:05:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:05:05.215714 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a581d1a1-ce66-4bc8-97d9-9a66e7d5bab8 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:05.328366 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a581d1a1-ce66-4bc8-97d9-9a66e7d5bab8 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:05.354422 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:05.360377 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e752469-f094-4f57-adbd-66eb581ddd7d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:05.362360 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e752469-f094-4f57-adbd-66eb581ddd7d None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:05.363280 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60374e9-e66f-4088-83f2-980a7880f93f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:05.364260 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60374e9-e66f-4088-83f2-980a7880f93f None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:05.364662 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60374e9-e66f-4088-83f2-980a7880f93f None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:05.364937 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60374e9-e66f-4088-83f2-980a7880f93f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:05.378276 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60374e9-e66f-4088-83f2-980a7880f93f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 19:05:05.383262 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-f60374e9-e66f-4088-83f2-980a7880f93f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 19:05:05.383262 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:05:05.383262 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:05:05.383262 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:05:05.383262 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:05:05.383262 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:05:05.383262 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:05:05.383262 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:05:05.383262 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:05:05.383262 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:05:05.383262 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:05:05.383262 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:05:05.383262 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 19:05:05.383262 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 19:05:05.383262 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 19:05:05.383262 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 19:05:05.384192 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 19:05:05.384192 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 19:05:05.384192 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 19:05:05.384192 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 19:05:05.384192 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:05:05.384192 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:05:05.384192 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:05:05.384192 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:05:05.384192 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:05:05.384192 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:05:05.384192 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:05:05.384192 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:05:05.384192 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:05:05.384192 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:05:05.384192 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:05:05.384192 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:05:05.385044 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:05:05.385044 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:05:05.385044 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:05:05.385044 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:05:05.385044 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 19:05:05.385044 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 19:05:05.385044 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 19:05:05.385044 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 19:05:05.385044 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 19:05:05.385044 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 19:05:05.385044 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 19:05:05.385044 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:05:05.385044 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 83/160] 10.222.0.61 () {56 vars in 1096 bytes} [Mon Aug 22 19:05:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:05:05.385689 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfcc0d0-f9e7-497c-a5a8-614cf4cc6e87 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:05.385754 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfcc0d0-f9e7-497c-a5a8-614cf4cc6e87 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:05.386476 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfcc0d0-f9e7-497c-a5a8-614cf4cc6e87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:05.406052 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-3cfcc0d0-f9e7-497c-a5a8-614cf4cc6e87 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:05:05.406052 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-3cfcc0d0-f9e7-497c-a5a8-614cf4cc6e87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:05:05.448019 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:05.474043 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 84/161] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:05:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 19:05:05.474883 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfcc0d0-f9e7-497c-a5a8-614cf4cc6e87 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:05.576668 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfcc0d0-f9e7-497c-a5a8-614cf4cc6e87 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:05.612631 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2a574d-b7d3-44d6-aa29-aa33f08fd778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:05.613292 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2a574d-b7d3-44d6-aa29-aa33f08fd778 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:05.613826 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2a574d-b7d3-44d6-aa29-aa33f08fd778 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:05.614098 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2a574d-b7d3-44d6-aa29-aa33f08fd778 None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:05.614773 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2a574d-b7d3-44d6-aa29-aa33f08fd778 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:05.615076 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2a574d-b7d3-44d6-aa29-aa33f08fd778 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:05.620250 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 78/162] 10.222.0.61 () {58 vars in 1282 bytes} [Mon Aug 22 19:05:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 607 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 22 19:05:05.620918 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2a574d-b7d3-44d6-aa29-aa33f08fd778 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:05.621270 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e752469-f094-4f57-adbd-66eb581ddd7d None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:05.633308 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 85/163] 10.222.0.61 () {56 vars in 1111 bytes} [Mon Aug 22 19:05:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:05.635160 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2a574d-b7d3-44d6-aa29-aa33f08fd778 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:05.636718 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:05.805677 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e752469-f094-4f57-adbd-66eb581ddd7d None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:05.845018 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2a574d-b7d3-44d6-aa29-aa33f08fd778 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:05.871676 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135f1f36-8edf-4b21-bf7d-f33c529f9b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:05.872135 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135f1f36-8edf-4b21-bf7d-f33c529f9b89 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:05.872413 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135f1f36-8edf-4b21-bf7d-f33c529f9b89 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:05.872698 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135f1f36-8edf-4b21-bf7d-f33c529f9b89 None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:05.873490 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135f1f36-8edf-4b21-bf7d-f33c529f9b89 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:05.873800 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135f1f36-8edf-4b21-bf7d-f33c529f9b89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:05.880326 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135f1f36-8edf-4b21-bf7d-f33c529f9b89 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:05.883979 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:05.894950 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 79/164] 10.222.0.61 () {56 vars in 1099 bytes} [Mon Aug 22 19:05:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:05.895863 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135f1f36-8edf-4b21-bf7d-f33c529f9b89 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:05.972953 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc1eca6-c4a8-42c6-900f-c09748a642af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:05.973600 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc1eca6-c4a8-42c6-900f-c09748a642af None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:05.973875 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc1eca6-c4a8-42c6-900f-c09748a642af None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:05.974143 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc1eca6-c4a8-42c6-900f-c09748a642af None admin] PATH_INFO: `/v3/projects/a4fe339e17324d15a22e51626818eecd` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:05.980536 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc1eca6-c4a8-42c6-900f-c09748a642af None admin] RBAC: Authorizing `identity:get_project(project_id=a4fe339e17324d15a22e51626818eecd)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:05.982029 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc1eca6-c4a8-42c6-900f-c09748a642af None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:05.983939 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 86/165] 10.222.0.61 () {56 vars in 1186 bytes} [Mon Aug 22 19:05:05 2022] GET /identity/v3/projects/a4fe339e17324d15a22e51626818eecd => generated 298 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:05.985189 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc1eca6-c4a8-42c6-900f-c09748a642af None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:06.065732 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:06.070179 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135f1f36-8edf-4b21-bf7d-f33c529f9b89 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:06.156127 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc1eca6-c4a8-42c6-900f-c09748a642af None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:06.156381 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c11fd23-56a3-4b22-8b20-ab57b9fd7462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:06.157041 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c11fd23-56a3-4b22-8b20-ab57b9fd7462 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:06.157319 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c11fd23-56a3-4b22-8b20-ab57b9fd7462 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:06.157591 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c11fd23-56a3-4b22-8b20-ab57b9fd7462 None admin] PATH_INFO: `/v3/users/a0683ebbf01f453fb5544a458adb02f4` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:06.160542 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c11fd23-56a3-4b22-8b20-ab57b9fd7462 None admin] RBAC: Authorizing `identity:get_user(user_id=a0683ebbf01f453fb5544a458adb02f4)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:06.162032 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c11fd23-56a3-4b22-8b20-ab57b9fd7462 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:06.163652 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 80/166] 10.222.0.61 () {56 vars in 1177 bytes} [Mon Aug 22 19:05:05 2022] GET /identity/v3/users/a0683ebbf01f453fb5544a458adb02f4 => generated 275 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:06.164492 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c11fd23-56a3-4b22-8b20-ab57b9fd7462 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:06.193794 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d595a5-4bc1-4a2e-8b45-a0edfe85b342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:06.194530 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d595a5-4bc1-4a2e-8b45-a0edfe85b342 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:06.194815 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d595a5-4bc1-4a2e-8b45-a0edfe85b342 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:06.195091 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d595a5-4bc1-4a2e-8b45-a0edfe85b342 None admin] PATH_INFO: `/v3/users/d9c7158eb3ce41ab8bb7a5d05da4a2f6` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:06.196322 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d595a5-4bc1-4a2e-8b45-a0edfe85b342 None admin] RBAC: Authorizing `identity:get_user(user_id=d9c7158eb3ce41ab8bb7a5d05da4a2f6)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:06.197819 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d595a5-4bc1-4a2e-8b45-a0edfe85b342 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:06.199499 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 87/167] 10.222.0.61 () {56 vars in 1177 bytes} [Mon Aug 22 19:05:05 2022] GET /identity/v3/users/d9c7158eb3ce41ab8bb7a5d05da4a2f6 => generated 247 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:06.200281 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d595a5-4bc1-4a2e-8b45-a0edfe85b342 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:06.308001 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:06.340927 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d595a5-4bc1-4a2e-8b45-a0edfe85b342 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:06.371089 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:06.382271 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c11fd23-56a3-4b22-8b20-ab57b9fd7462 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:06.408190 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7421675-ff24-427a-8122-f99c7659c9c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:06.408751 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7421675-ff24-427a-8122-f99c7659c9c7 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:06.409031 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7421675-ff24-427a-8122-f99c7659c9c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:06.409306 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7421675-ff24-427a-8122-f99c7659c9c7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:06.430942 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7421675-ff24-427a-8122-f99c7659c9c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 19:05:06.439732 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-b7421675-ff24-427a-8122-f99c7659c9c7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 22 19:05:06.439732 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:05:06.439732 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:05:06.439732 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:05:06.439732 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:05:06.439732 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:05:06.439732 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:05:06.439732 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:05:06.439732 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:05:06.439732 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:05:06.439732 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:05:06.439732 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:05:06.439732 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 19:05:06.439732 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 19:05:06.439732 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 19:05:06.439732 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 19:05:06.441979 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 19:05:06.441979 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 19:05:06.441979 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 19:05:06.441979 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 19:05:06.441979 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:05:06.441979 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:05:06.441979 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:05:06.441979 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:05:06.441979 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:05:06.441979 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:05:06.441979 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:05:06.441979 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:05:06.441979 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:05:06.441979 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:05:06.441979 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:05:06.441979 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:05:06.443997 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:05:06.443997 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:05:06.443997 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:05:06.443997 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:05:06.443997 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 19:05:06.443997 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 19:05:06.443997 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 19:05:06.443997 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 19:05:06.443997 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 19:05:06.443997 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 19:05:06.443997 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 22 19:05:06.443997 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:05:06.445525 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0dec4f-8b88-474e-ab62-4cb97e4d8edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:06.445525 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0dec4f-8b88-474e-ab62-4cb97e4d8edd None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:06.445838 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0dec4f-8b88-474e-ab62-4cb97e4d8edd None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:06.445996 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0dec4f-8b88-474e-ab62-4cb97e4d8edd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:06.450838 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0dec4f-8b88-474e-ab62-4cb97e4d8edd None admin] RBAC: Adding query filter params (role.id=74a40bc819cb42eea7e966a4b2a016f1, scope.project.id=a4fe339e17324d15a22e51626818eecd, user.id=a0683ebbf01f453fb5544a458adb02f4) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:06.450838 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0dec4f-8b88-474e-ab62-4cb97e4d8edd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:06.450838 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0dec4f-8b88-474e-ab62-4cb97e4d8edd None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:06.451824 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 88/168] 10.222.0.61 () {56 vars in 1099 bytes} [Mon Aug 22 19:05:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:05:06.451824 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7421675-ff24-427a-8122-f99c7659c9c7 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:06.463196 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 81/169] 10.222.0.61 () {56 vars in 1374 bytes} [Mon Aug 22 19:05:06 2022] GET /identity/v3/role_assignments?user.id=a0683ebbf01f453fb5544a458adb02f4&scope.project.id=a4fe339e17324d15a22e51626818eecd&role.id=74a40bc819cb42eea7e966a4b2a016f1 => generated 608 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:06.476220 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e250165a-35db-46b4-a70c-35dafc5e8124 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:06.476554 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e250165a-35db-46b4-a70c-35dafc5e8124 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:06.476762 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e250165a-35db-46b4-a70c-35dafc5e8124 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:06.478322 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 82/170] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:05:06 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:05:06.478948 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e250165a-35db-46b4-a70c-35dafc5e8124 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:06.671492 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7421675-ff24-427a-8122-f99c7659c9c7 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:06.688040 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e250165a-35db-46b4-a70c-35dafc5e8124 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:06.746342 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d8d91a-6127-4d9d-a038-17b56130a8e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:06.746963 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d8d91a-6127-4d9d-a038-17b56130a8e1 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:06.747241 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d8d91a-6127-4d9d-a038-17b56130a8e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:06.747515 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d8d91a-6127-4d9d-a038-17b56130a8e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:06.747997 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:06.760886 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24808a3c-366f-4d99-b7ee-41366712a217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:06.762818 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24808a3c-366f-4d99-b7ee-41366712a217 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:06.763134 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24808a3c-366f-4d99-b7ee-41366712a217 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:06.763411 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24808a3c-366f-4d99-b7ee-41366712a217 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:06.831437 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:06.898189 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d8d91a-6127-4d9d-a038-17b56130a8e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:06.899717 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d8d91a-6127-4d9d-a038-17b56130a8e1 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:06.902425 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24808a3c-366f-4d99-b7ee-41366712a217 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:06.903907 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24808a3c-366f-4d99-b7ee-41366712a217 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:07.059586 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 89/171] 10.222.0.61 () {58 vars in 1282 bytes} [Mon Aug 22 19:05:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 608 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 19:05:07.063013 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d8d91a-6127-4d9d-a038-17b56130a8e1 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:07.072119 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 83/172] 10.222.0.61 () {58 vars in 1282 bytes} [Mon Aug 22 19:05:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 593 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 19:05:07.073530 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15282881-0c26-4126-b9f4-17404149269c None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:07.074048 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15282881-0c26-4126-b9f4-17404149269c None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:07.074699 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15282881-0c26-4126-b9f4-17404149269c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:07.094117 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-15282881-0c26-4126-b9f4-17404149269c None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:05:07.095397 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-15282881-0c26-4126-b9f4-17404149269c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:05:07.120672 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:07.142854 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:07.168776 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 84/173] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:05:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 19:05:07.173311 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8994795-e756-4c47-9d8b-f3127a6426f4 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:07.173842 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8994795-e756-4c47-9d8b-f3127a6426f4 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:07.174332 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8994795-e756-4c47-9d8b-f3127a6426f4 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:07.174880 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d8d91a-6127-4d9d-a038-17b56130a8e1 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:07.176412 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 85/174] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:05:07 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:05:07.177584 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb70d2f7-572c-44e1-981a-44d194d6bb61 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:07.177941 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb70d2f7-572c-44e1-981a-44d194d6bb61 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:07.178289 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb70d2f7-572c-44e1-981a-44d194d6bb61 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:07.183593 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 86/175] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:05:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:05:07.183593 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb70d2f7-572c-44e1-981a-44d194d6bb61 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:07.219379 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eb004e-36fc-45d6-966b-40156fefbb0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:07.220035 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eb004e-36fc-45d6-966b-40156fefbb0f None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:07.220313 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eb004e-36fc-45d6-966b-40156fefbb0f None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:07.220581 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eb004e-36fc-45d6-966b-40156fefbb0f None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:07.221247 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80eb004e-36fc-45d6-966b-40156fefbb0f None admin] RBAC: Adding query filter params (name=member) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:07.221548 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80eb004e-36fc-45d6-966b-40156fefbb0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:07.223336 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80eb004e-36fc-45d6-966b-40156fefbb0f None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:07.232078 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 90/176] 10.222.0.61 () {56 vars in 1101 bytes} [Mon Aug 22 19:05:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:07.232860 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eb004e-36fc-45d6-966b-40156fefbb0f None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:07.294217 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb70d2f7-572c-44e1-981a-44d194d6bb61 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:07.331926 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1292fb-2412-432c-98d5-fdd1b97d51da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:07.332872 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1292fb-2412-432c-98d5-fdd1b97d51da None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:07.333340 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1292fb-2412-432c-98d5-fdd1b97d51da None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:07.333790 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1292fb-2412-432c-98d5-fdd1b97d51da None admin] PATH_INFO: `/v3/projects/a4fe339e17324d15a22e51626818eecd` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:07.335365 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1292fb-2412-432c-98d5-fdd1b97d51da None admin] RBAC: Authorizing `identity:get_project(project_id=a4fe339e17324d15a22e51626818eecd)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:07.337029 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1292fb-2412-432c-98d5-fdd1b97d51da None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:07.338865 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 87/177] 10.222.0.61 () {56 vars in 1186 bytes} [Mon Aug 22 19:05:07 2022] GET /identity/v3/projects/a4fe339e17324d15a22e51626818eecd => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:07.340170 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b5253c-6401-44ec-8c13-ee60d4d3ec00 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:07.340446 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b5253c-6401-44ec-8c13-ee60d4d3ec00 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:07.340718 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b5253c-6401-44ec-8c13-ee60d4d3ec00 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:07.346011 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 88/178] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:05:07 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:05:07.346011 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b34c673-f365-4d68-b144-96bc5eb2d2b8 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:07.346011 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b34c673-f365-4d68-b144-96bc5eb2d2b8 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:07.346011 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b34c673-f365-4d68-b144-96bc5eb2d2b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:07.351188 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eb004e-36fc-45d6-966b-40156fefbb0f None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:07.361592 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-9b34c673-f365-4d68-b144-96bc5eb2d2b8 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:05:07.362533 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-9b34c673-f365-4d68-b144-96bc5eb2d2b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:05:07.383596 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:07.386149 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:07.390049 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f4a8fc-65c2-4ce4-b001-ebbec6cbf04f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:07.390902 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f4a8fc-65c2-4ce4-b001-ebbec6cbf04f None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:07.391350 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f4a8fc-65c2-4ce4-b001-ebbec6cbf04f None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:07.391628 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f4a8fc-65c2-4ce4-b001-ebbec6cbf04f None admin] PATH_INFO: `/v3/projects/a4fe339e17324d15a22e51626818eecd` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:07.392783 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f4a8fc-65c2-4ce4-b001-ebbec6cbf04f None admin] RBAC: Authorizing `identity:get_project(project_id=a4fe339e17324d15a22e51626818eecd)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:07.394221 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f4a8fc-65c2-4ce4-b001-ebbec6cbf04f None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:07.395902 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 91/179] 10.222.0.61 () {56 vars in 1186 bytes} [Mon Aug 22 19:05:07 2022] GET /identity/v3/projects/a4fe339e17324d15a22e51626818eecd => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:07.397003 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b769f935-77cb-4a0c-b9af-a115c9c41ace None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:07.397283 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b769f935-77cb-4a0c-b9af-a115c9c41ace None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:07.397556 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b769f935-77cb-4a0c-b9af-a115c9c41ace None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:07.415353 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-b769f935-77cb-4a0c-b9af-a115c9c41ace None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:05:07.418374 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-b769f935-77cb-4a0c-b9af-a115c9c41ace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:05:07.443406 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 89/180] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:05:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 19:05:07.447485 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b34c673-f365-4d68-b144-96bc5eb2d2b8 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:07.489634 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 92/181] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:05:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 19:05:07.492600 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b769f935-77cb-4a0c-b9af-a115c9c41ace None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:07.574662 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b34c673-f365-4d68-b144-96bc5eb2d2b8 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:07.612609 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b88b04-a4d9-4180-bb32-d071c807283d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:07.613175 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b88b04-a4d9-4180-bb32-d071c807283d None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:07.613730 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b88b04-a4d9-4180-bb32-d071c807283d None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:07.613991 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b88b04-a4d9-4180-bb32-d071c807283d None admin] PATH_INFO: `/v3/users/a0683ebbf01f453fb5544a458adb02f4` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:07.615187 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b88b04-a4d9-4180-bb32-d071c807283d None admin] RBAC: Authorizing `identity:get_user(user_id=a0683ebbf01f453fb5544a458adb02f4)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:07.616173 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b769f935-77cb-4a0c-b9af-a115c9c41ace None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:07.616787 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b88b04-a4d9-4180-bb32-d071c807283d None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:07.618391 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 90/182] 10.222.0.61 () {56 vars in 1177 bytes} [Mon Aug 22 19:05:07 2022] GET /identity/v3/users/a0683ebbf01f453fb5544a458adb02f4 => generated 275 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:07.624278 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62985407-ba97-47fa-a295-2a56ccbe4f39 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:07.624570 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62985407-ba97-47fa-a295-2a56ccbe4f39 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:07.624841 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62985407-ba97-47fa-a295-2a56ccbe4f39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:07.643503 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-62985407-ba97-47fa-a295-2a56ccbe4f39 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:05:07.644778 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-62985407-ba97-47fa-a295-2a56ccbe4f39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:05:07.650264 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:07.657569 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b591766-fb6d-43a3-8666-6ef82609dbe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:07.658768 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b591766-fb6d-43a3-8666-6ef82609dbe4 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:07.659252 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b591766-fb6d-43a3-8666-6ef82609dbe4 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:07.659823 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b591766-fb6d-43a3-8666-6ef82609dbe4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:07.661196 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b591766-fb6d-43a3-8666-6ef82609dbe4 None admin] RBAC: Adding query filter params (role.id=9f0952608a704dfca5fdbe4046331f21, scope.project.id=a4fe339e17324d15a22e51626818eecd, user.id=a0683ebbf01f453fb5544a458adb02f4) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:07.661716 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b591766-fb6d-43a3-8666-6ef82609dbe4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:07.663463 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b591766-fb6d-43a3-8666-6ef82609dbe4 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:07.664088 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:07.670906 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 93/183] 10.222.0.61 () {56 vars in 1374 bytes} [Mon Aug 22 19:05:07 2022] GET /identity/v3/role_assignments?user.id=a0683ebbf01f453fb5544a458adb02f4&scope.project.id=a4fe339e17324d15a22e51626818eecd&role.id=9f0952608a704dfca5fdbe4046331f21 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:07.672837 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b591766-fb6d-43a3-8666-6ef82609dbe4 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:07.758230 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 91/184] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:05:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 19:05:07.760748 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62985407-ba97-47fa-a295-2a56ccbe4f39 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:07.831865 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b591766-fb6d-43a3-8666-6ef82609dbe4 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:07.870507 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4d02bc-66df-4bdd-8981-fd0707a8db61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:07.871439 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4d02bc-66df-4bdd-8981-fd0707a8db61 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:07.871996 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4d02bc-66df-4bdd-8981-fd0707a8db61 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:07.872393 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4d02bc-66df-4bdd-8981-fd0707a8db61 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:07.873200 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4d02bc-66df-4bdd-8981-fd0707a8db61 None admin] RBAC: Adding query filter params (role.id=163c30c3b3c246beb849ce176f02d9da, scope.project.id=a4fe339e17324d15a22e51626818eecd, user.id=d9c7158eb3ce41ab8bb7a5d05da4a2f6) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:07.873451 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4d02bc-66df-4bdd-8981-fd0707a8db61 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:07.876030 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4d02bc-66df-4bdd-8981-fd0707a8db61 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:07.882525 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 94/185] 10.222.0.61 () {56 vars in 1374 bytes} [Mon Aug 22 19:05:07 2022] GET /identity/v3/role_assignments?user.id=d9c7158eb3ce41ab8bb7a5d05da4a2f6&scope.project.id=a4fe339e17324d15a22e51626818eecd&role.id=163c30c3b3c246beb849ce176f02d9da => generated 608 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:07.883321 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4d02bc-66df-4bdd-8981-fd0707a8db61 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:07.894295 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62985407-ba97-47fa-a295-2a56ccbe4f39 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:07.947204 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032be00a-009e-4744-bad7-0dc24f5fbade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:07.948017 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032be00a-009e-4744-bad7-0dc24f5fbade None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:07.948428 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032be00a-009e-4744-bad7-0dc24f5fbade None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:07.949581 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032be00a-009e-4744-bad7-0dc24f5fbade None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:07.965939 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032be00a-009e-4744-bad7-0dc24f5fbade None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 19:05:07.973448 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-032be00a-009e-4744-bad7-0dc24f5fbade None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 19:05:07.973448 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:05:07.973448 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:05:07.973448 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:05:07.973448 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:05:07.973448 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:05:07.973448 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:05:07.973448 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:05:07.973448 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:05:07.973448 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:05:07.973448 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:05:07.973448 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:05:07.973448 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 19:05:07.973448 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 19:05:07.973448 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 19:05:07.973448 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 19:05:07.974461 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 19:05:07.974461 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 19:05:07.974461 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 19:05:07.974461 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 19:05:07.974461 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:05:07.974461 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:05:07.974461 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:05:07.974461 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:05:07.974461 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:05:07.974461 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:05:07.974461 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:05:07.974461 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:05:07.974461 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:05:07.974461 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:05:07.974461 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:05:07.974461 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:05:07.975276 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:05:07.975276 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:05:07.975276 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:05:07.975276 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:05:07.975276 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 19:05:07.975276 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 19:05:07.975276 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 19:05:07.975276 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 19:05:07.975276 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 19:05:07.975276 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 19:05:07.975276 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 19:05:07.975276 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:05:07.975276 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 92/186] 10.222.0.61 () {56 vars in 1096 bytes} [Mon Aug 22 19:05:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:05:07.976312 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032be00a-009e-4744-bad7-0dc24f5fbade None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:07.978101 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:08.001987 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:08.040839 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4d02bc-66df-4bdd-8981-fd0707a8db61 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:08.084095 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152023f2-2dd5-4e43-94c8-36a68e60c91d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:08.084659 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152023f2-2dd5-4e43-94c8-36a68e60c91d None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:08.084935 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152023f2-2dd5-4e43-94c8-36a68e60c91d None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:08.085201 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152023f2-2dd5-4e43-94c8-36a68e60c91d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:08.101926 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032be00a-009e-4744-bad7-0dc24f5fbade None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:08.148260 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55614bdd-e266-4741-b839-66179ab827a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:08.148893 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55614bdd-e266-4741-b839-66179ab827a1 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:08.149166 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55614bdd-e266-4741-b839-66179ab827a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:08.149451 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55614bdd-e266-4741-b839-66179ab827a1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:08.160380 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55614bdd-e266-4741-b839-66179ab827a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 19:05:08.165189 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-55614bdd-e266-4741-b839-66179ab827a1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 22 19:05:08.165189 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:05:08.165189 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:05:08.165189 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:05:08.165189 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:05:08.165189 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:05:08.165189 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:05:08.165189 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:05:08.165189 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:05:08.165189 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:05:08.165189 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:05:08.165189 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:05:08.165189 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 19:05:08.165189 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 19:05:08.165189 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 19:05:08.165189 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 19:05:08.166104 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 19:05:08.166104 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 19:05:08.166104 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 19:05:08.166104 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 19:05:08.166104 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:05:08.166104 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:05:08.166104 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:05:08.166104 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:05:08.166104 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:05:08.166104 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:05:08.166104 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:05:08.166104 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:05:08.166104 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:05:08.166104 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:05:08.166104 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:05:08.166104 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:05:08.166994 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:05:08.166994 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:05:08.166994 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:05:08.166994 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:05:08.166994 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 19:05:08.166994 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 19:05:08.166994 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 19:05:08.166994 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 19:05:08.166994 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 19:05:08.166994 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 19:05:08.166994 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 22 19:05:08.166994 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:05:08.166994 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 93/187] 10.222.0.61 () {56 vars in 1099 bytes} [Mon Aug 22 19:05:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:05:08.167725 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152023f2-2dd5-4e43-94c8-36a68e60c91d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:08.168053 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152023f2-2dd5-4e43-94c8-36a68e60c91d None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:08.171752 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55614bdd-e266-4741-b839-66179ab827a1 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:08.224325 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:08.246011 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:08.268164 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 95/188] 10.222.0.61 () {58 vars in 1282 bytes} [Mon Aug 22 19:05:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 19:05:08.268995 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152023f2-2dd5-4e43-94c8-36a68e60c91d None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:08.274886 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55614bdd-e266-4741-b839-66179ab827a1 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:08.309557 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3f4b1c-a29d-4eb5-a49e-da7aabf81822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:08.310180 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3f4b1c-a29d-4eb5-a49e-da7aabf81822 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:08.310452 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3f4b1c-a29d-4eb5-a49e-da7aabf81822 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:08.310720 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3f4b1c-a29d-4eb5-a49e-da7aabf81822 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:08.320833 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3f4b1c-a29d-4eb5-a49e-da7aabf81822 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 19:05:08.325652 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-5d3f4b1c-a29d-4eb5-a49e-da7aabf81822 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 19:05:08.325652 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:05:08.325652 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:05:08.325652 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:05:08.325652 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:05:08.325652 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:05:08.325652 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:05:08.325652 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:05:08.325652 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:05:08.325652 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:05:08.325652 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:05:08.325652 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:05:08.325652 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 19:05:08.325652 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 19:05:08.325652 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 19:05:08.325652 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 19:05:08.326526 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 19:05:08.326526 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 19:05:08.326526 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 19:05:08.326526 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 19:05:08.326526 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:05:08.326526 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:05:08.326526 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:05:08.326526 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:05:08.326526 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:05:08.326526 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:05:08.326526 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:05:08.326526 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:05:08.326526 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:05:08.326526 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:05:08.326526 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:05:08.326526 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:05:08.332593 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:05:08.332593 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:05:08.332593 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:05:08.332593 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:05:08.332593 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 19:05:08.332593 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 19:05:08.332593 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 19:05:08.332593 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 19:05:08.332593 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 19:05:08.332593 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 19:05:08.332593 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 19:05:08.332593 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:05:08.332593 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 94/189] 10.222.0.61 () {56 vars in 1114 bytes} [Mon Aug 22 19:05:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:05:08.332593 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3f4b1c-a29d-4eb5-a49e-da7aabf81822 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:08.368039 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152023f2-2dd5-4e43-94c8-36a68e60c91d None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:08.402418 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b7ef5f-cb22-4069-bf1d-6b201969944c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:08.402934 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b7ef5f-cb22-4069-bf1d-6b201969944c None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:08.403598 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b7ef5f-cb22-4069-bf1d-6b201969944c None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:08.403902 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b7ef5f-cb22-4069-bf1d-6b201969944c None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:08.404564 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b7ef5f-cb22-4069-bf1d-6b201969944c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:08.404866 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b7ef5f-cb22-4069-bf1d-6b201969944c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:08.407182 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b7ef5f-cb22-4069-bf1d-6b201969944c None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:08.415032 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 96/190] 10.222.0.61 () {56 vars in 1099 bytes} [Mon Aug 22 19:05:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:08.415844 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b7ef5f-cb22-4069-bf1d-6b201969944c None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:08.429502 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3f4b1c-a29d-4eb5-a49e-da7aabf81822 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:08.440156 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:08.462987 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90009d72-0440-4321-8a7b-d9b051fe8f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:08.463622 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90009d72-0440-4321-8a7b-d9b051fe8f20 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:08.463934 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90009d72-0440-4321-8a7b-d9b051fe8f20 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:08.464203 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90009d72-0440-4321-8a7b-d9b051fe8f20 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:08.465321 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90009d72-0440-4321-8a7b-d9b051fe8f20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:08.466770 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90009d72-0440-4321-8a7b-d9b051fe8f20 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:08.468364 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 95/191] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:05:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:08.469660 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90009d72-0440-4321-8a7b-d9b051fe8f20 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:08.472122 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:08.519231 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b7ef5f-cb22-4069-bf1d-6b201969944c None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:08.551373 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c66067a-e62d-448f-88b7-1f5622599559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:08.552016 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c66067a-e62d-448f-88b7-1f5622599559 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:08.552297 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c66067a-e62d-448f-88b7-1f5622599559 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:08.552568 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c66067a-e62d-448f-88b7-1f5622599559 None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:08.553233 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c66067a-e62d-448f-88b7-1f5622599559 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:08.553535 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c66067a-e62d-448f-88b7-1f5622599559 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:08.554929 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c66067a-e62d-448f-88b7-1f5622599559 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:08.563406 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 97/192] 10.222.0.61 () {56 vars in 1101 bytes} [Mon Aug 22 19:05:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:08.567884 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c66067a-e62d-448f-88b7-1f5622599559 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:08.570859 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90009d72-0440-4321-8a7b-d9b051fe8f20 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:08.604843 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d0740a-2d26-4bd3-bda7-6296b0308e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:08.605467 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d0740a-2d26-4bd3-bda7-6296b0308e5e None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:08.605733 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d0740a-2d26-4bd3-bda7-6296b0308e5e None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:08.606002 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d0740a-2d26-4bd3-bda7-6296b0308e5e None admin] PATH_INFO: `/v3/projects/f6d87df75a414a59b78d9ce3488c7059` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:08.607124 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d0740a-2d26-4bd3-bda7-6296b0308e5e None admin] RBAC: Authorizing `identity:get_project(project_id=f6d87df75a414a59b78d9ce3488c7059)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:08.608616 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d0740a-2d26-4bd3-bda7-6296b0308e5e None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:08.610278 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 96/193] 10.222.0.61 () {56 vars in 1186 bytes} [Mon Aug 22 19:05:08 2022] GET /identity/v3/projects/f6d87df75a414a59b78d9ce3488c7059 => generated 312 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:08.611030 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d0740a-2d26-4bd3-bda7-6296b0308e5e None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:08.654589 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:08.673233 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c66067a-e62d-448f-88b7-1f5622599559 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:08.680750 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:08.706811 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cca36b0-9064-40dd-9758-a8c3cda4afd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:08.707414 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cca36b0-9064-40dd-9758-a8c3cda4afd6 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:08.707704 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cca36b0-9064-40dd-9758-a8c3cda4afd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:08.707978 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cca36b0-9064-40dd-9758-a8c3cda4afd6 None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:08.708644 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cca36b0-9064-40dd-9758-a8c3cda4afd6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:08.708941 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cca36b0-9064-40dd-9758-a8c3cda4afd6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:08.710298 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cca36b0-9064-40dd-9758-a8c3cda4afd6 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:08.712791 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d0740a-2d26-4bd3-bda7-6296b0308e5e None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:08.718314 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 98/194] 10.222.0.61 () {56 vars in 1111 bytes} [Mon Aug 22 19:05:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:08.719099 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cca36b0-9064-40dd-9758-a8c3cda4afd6 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:08.747508 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb5a1b0-a6d1-4723-8fdf-6ab6db0dc57a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:08.748164 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb5a1b0-a6d1-4723-8fdf-6ab6db0dc57a None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:08.748450 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb5a1b0-a6d1-4723-8fdf-6ab6db0dc57a None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:08.748723 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb5a1b0-a6d1-4723-8fdf-6ab6db0dc57a None admin] PATH_INFO: `/v3/users/61d1f8624beb408cb3bd4cbbde27c9bc` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:08.763997 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb5a1b0-a6d1-4723-8fdf-6ab6db0dc57a None admin] RBAC: Authorizing `identity:get_user(user_id=61d1f8624beb408cb3bd4cbbde27c9bc)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:08.765458 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb5a1b0-a6d1-4723-8fdf-6ab6db0dc57a None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:08.767611 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 97/195] 10.222.0.61 () {56 vars in 1177 bytes} [Mon Aug 22 19:05:08 2022] GET /identity/v3/users/61d1f8624beb408cb3bd4cbbde27c9bc => generated 283 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:08.767611 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb5a1b0-a6d1-4723-8fdf-6ab6db0dc57a None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:08.823171 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cca36b0-9064-40dd-9758-a8c3cda4afd6 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:08.855526 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab74a667-6754-43bf-95f3-db1971d760c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:08.856218 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab74a667-6754-43bf-95f3-db1971d760c3 None admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:08.856491 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab74a667-6754-43bf-95f3-db1971d760c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:08.856762 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab74a667-6754-43bf-95f3-db1971d760c3 None admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:08.857391 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab74a667-6754-43bf-95f3-db1971d760c3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:08.858813 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab74a667-6754-43bf-95f3-db1971d760c3 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:08.864952 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb5a1b0-a6d1-4723-8fdf-6ab6db0dc57a None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:08.874996 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:08.880587 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 99/196] 10.222.0.61 () {60 vars in 1132 bytes} [Mon Aug 22 19:05:08 2022] POST /identity/v3/users => generated 297 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:05:08.881373 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab74a667-6754-43bf-95f3-db1971d760c3 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:08.903816 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:08.913616 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f660eb9c-5d6a-4622-a523-32130a0c70f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:08.914460 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f660eb9c-5d6a-4622-a523-32130a0c70f1 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:08.914906 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f660eb9c-5d6a-4622-a523-32130a0c70f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:08.915614 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f660eb9c-5d6a-4622-a523-32130a0c70f1 None admin] PATH_INFO: `/v3/users/14147cb34fc743e89b373b1a740217f8` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:08.916896 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f660eb9c-5d6a-4622-a523-32130a0c70f1 None admin] RBAC: Authorizing `identity:get_user(user_id=14147cb34fc743e89b373b1a740217f8)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:08.918396 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f660eb9c-5d6a-4622-a523-32130a0c70f1 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:08.925358 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 98/197] 10.222.0.61 () {56 vars in 1177 bytes} [Mon Aug 22 19:05:08 2022] GET /identity/v3/users/14147cb34fc743e89b373b1a740217f8 => generated 289 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:08.925358 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f660eb9c-5d6a-4622-a523-32130a0c70f1 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:09.008173 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab74a667-6754-43bf-95f3-db1971d760c3 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:09.041455 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f660eb9c-5d6a-4622-a523-32130a0c70f1 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:09.049356 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8198bc-d267-4db6-a7ba-1d0870950f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:09.050060 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8198bc-d267-4db6-a7ba-1d0870950f6f None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:09.050245 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8198bc-d267-4db6-a7ba-1d0870950f6f None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:09.050511 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8198bc-d267-4db6-a7ba-1d0870950f6f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:09.051248 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8198bc-d267-4db6-a7ba-1d0870950f6f None admin] RBAC: Adding query filter params (role.id=74a40bc819cb42eea7e966a4b2a016f1, scope.project.id=f6d87df75a414a59b78d9ce3488c7059, user.id=a0683ebbf01f453fb5544a458adb02f4) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:09.051576 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8198bc-d267-4db6-a7ba-1d0870950f6f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:09.053719 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8198bc-d267-4db6-a7ba-1d0870950f6f None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:09.059845 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 100/198] 10.222.0.61 () {56 vars in 1374 bytes} [Mon Aug 22 19:05:08 2022] GET /identity/v3/role_assignments?user.id=a0683ebbf01f453fb5544a458adb02f4&scope.project.id=f6d87df75a414a59b78d9ce3488c7059&role.id=74a40bc819cb42eea7e966a4b2a016f1 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:09.060721 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8198bc-d267-4db6-a7ba-1d0870950f6f None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:09.079526 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b5a8a6-5026-4a4a-8220-859035317499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:09.080300 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b5a8a6-5026-4a4a-8220-859035317499 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:09.080588 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b5a8a6-5026-4a4a-8220-859035317499 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:09.080853 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b5a8a6-5026-4a4a-8220-859035317499 None admin] PATH_INFO: `/v3/users/61d1f8624beb408cb3bd4cbbde27c9bc` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:09.082105 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b5a8a6-5026-4a4a-8220-859035317499 None admin] RBAC: Authorizing `identity:get_user(user_id=61d1f8624beb408cb3bd4cbbde27c9bc)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:09.083630 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b5a8a6-5026-4a4a-8220-859035317499 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:09.085554 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 99/199] 10.222.0.61 () {56 vars in 1177 bytes} [Mon Aug 22 19:05:08 2022] GET /identity/v3/users/61d1f8624beb408cb3bd4cbbde27c9bc => generated 283 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:09.086209 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b5a8a6-5026-4a4a-8220-859035317499 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:09.151552 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:09.186312 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:09.208626 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8198bc-d267-4db6-a7ba-1d0870950f6f None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:09.228853 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b5a8a6-5026-4a4a-8220-859035317499 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:09.260625 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375719ff-119a-42d7-8ab3-b8051f0b47f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:09.261308 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375719ff-119a-42d7-8ab3-b8051f0b47f4 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:09.261529 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375719ff-119a-42d7-8ab3-b8051f0b47f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:09.261796 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375719ff-119a-42d7-8ab3-b8051f0b47f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:09.270097 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80a6f15-dbf7-4285-856d-a5c6f3df4f99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:09.270971 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80a6f15-dbf7-4285-856d-a5c6f3df4f99 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:09.271420 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80a6f15-dbf7-4285-856d-a5c6f3df4f99 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:09.271890 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80a6f15-dbf7-4285-856d-a5c6f3df4f99 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:09.340096 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375719ff-119a-42d7-8ab3-b8051f0b47f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:09.341700 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375719ff-119a-42d7-8ab3-b8051f0b47f4 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:09.356315 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80a6f15-dbf7-4285-856d-a5c6f3df4f99 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:09.358425 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80a6f15-dbf7-4285-856d-a5c6f3df4f99 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:09.418633 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:09.426344 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:09.465813 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 101/200] 10.222.0.61 () {58 vars in 1282 bytes} [Mon Aug 22 19:05:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 19:05:09.466784 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375719ff-119a-42d7-8ab3-b8051f0b47f4 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:09.472036 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 100/201] 10.222.0.61 () {58 vars in 1282 bytes} [Mon Aug 22 19:05:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 19:05:09.472886 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80a6f15-dbf7-4285-856d-a5c6f3df4f99 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:09.582181 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375719ff-119a-42d7-8ab3-b8051f0b47f4 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:09.583292 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80a6f15-dbf7-4285-856d-a5c6f3df4f99 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:09.619998 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee748001-f2e8-4b95-8f28-6ff10f1b2a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:09.620760 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee748001-f2e8-4b95-8f28-6ff10f1b2a4a None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:09.621050 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee748001-f2e8-4b95-8f28-6ff10f1b2a4a None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:09.621153 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f1140a-9e2d-483c-8569-f92c6d318fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:09.621596 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee748001-f2e8-4b95-8f28-6ff10f1b2a4a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:09.621802 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f1140a-9e2d-483c-8569-f92c6d318fce None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:09.622034 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f1140a-9e2d-483c-8569-f92c6d318fce None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:09.622310 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f1140a-9e2d-483c-8569-f92c6d318fce None admin] PATH_INFO: `/v3/projects/47adea686aa0431fb9766ac68f4bf165` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:09.623464 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f1140a-9e2d-483c-8569-f92c6d318fce None admin] RBAC: Authorizing `identity:get_project(project_id=47adea686aa0431fb9766ac68f4bf165)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:09.625222 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f1140a-9e2d-483c-8569-f92c6d318fce None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:09.627205 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 101/202] 10.222.0.61 () {56 vars in 1186 bytes} [Mon Aug 22 19:05:09 2022] GET /identity/v3/projects/47adea686aa0431fb9766ac68f4bf165 => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:09.628082 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f1140a-9e2d-483c-8569-f92c6d318fce None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:09.655267 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:09.663039 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:09.697734 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee748001-f2e8-4b95-8f28-6ff10f1b2a4a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:09.699831 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee748001-f2e8-4b95-8f28-6ff10f1b2a4a None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:09.739210 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f1140a-9e2d-483c-8569-f92c6d318fce None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:09.776423 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755dea3c-964d-4750-aa34-b00057728ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:09.777065 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755dea3c-964d-4750-aa34-b00057728ffb None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:09.777349 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755dea3c-964d-4750-aa34-b00057728ffb None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:09.777624 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755dea3c-964d-4750-aa34-b00057728ffb None admin] PATH_INFO: `/v3/projects/a4fe339e17324d15a22e51626818eecd` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:09.778772 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755dea3c-964d-4750-aa34-b00057728ffb None admin] RBAC: Authorizing `identity:get_project(project_id=a4fe339e17324d15a22e51626818eecd)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:09.780259 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755dea3c-964d-4750-aa34-b00057728ffb None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:09.782716 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 102/203] 10.222.0.61 () {56 vars in 1186 bytes} [Mon Aug 22 19:05:09 2022] GET /identity/v3/projects/a4fe339e17324d15a22e51626818eecd => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:09.783483 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755dea3c-964d-4750-aa34-b00057728ffb None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:09.816357 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 102/204] 10.222.0.61 () {58 vars in 1282 bytes} [Mon Aug 22 19:05:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 19:05:09.817742 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee748001-f2e8-4b95-8f28-6ff10f1b2a4a None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:09.884099 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:09.887924 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:09.890758 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755dea3c-964d-4750-aa34-b00057728ffb None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:09.925824 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c928f9c-fb19-4f59-ab5a-10cb884e8975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:09.926364 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c928f9c-fb19-4f59-ab5a-10cb884e8975 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:09.926646 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c928f9c-fb19-4f59-ab5a-10cb884e8975 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:09.926923 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c928f9c-fb19-4f59-ab5a-10cb884e8975 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:09.927677 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c928f9c-fb19-4f59-ab5a-10cb884e8975 None admin] RBAC: Adding query filter params (role.id=163c30c3b3c246beb849ce176f02d9da, scope.project.id=47adea686aa0431fb9766ac68f4bf165, user.id=61d1f8624beb408cb3bd4cbbde27c9bc) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:09.927993 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c928f9c-fb19-4f59-ab5a-10cb884e8975 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:09.929190 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee748001-f2e8-4b95-8f28-6ff10f1b2a4a None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:09.929428 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c928f9c-fb19-4f59-ab5a-10cb884e8975 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:09.935395 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 103/205] 10.222.0.61 () {56 vars in 1374 bytes} [Mon Aug 22 19:05:09 2022] GET /identity/v3/role_assignments?user.id=61d1f8624beb408cb3bd4cbbde27c9bc&scope.project.id=47adea686aa0431fb9766ac68f4bf165&role.id=163c30c3b3c246beb849ce176f02d9da => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:09.936235 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c928f9c-fb19-4f59-ab5a-10cb884e8975 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:09.970588 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65d5fc7-69ac-41f7-bc7d-a6565e658f3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:09.971748 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65d5fc7-69ac-41f7-bc7d-a6565e658f3b None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:09.972353 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65d5fc7-69ac-41f7-bc7d-a6565e658f3b None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:09.972835 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65d5fc7-69ac-41f7-bc7d-a6565e658f3b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:09.973871 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65d5fc7-69ac-41f7-bc7d-a6565e658f3b None admin] RBAC: Adding query filter params (role.id=3f7ffde3fddd46caa6e49c24a6f07673, scope.project.id=a4fe339e17324d15a22e51626818eecd, user.id=14147cb34fc743e89b373b1a740217f8) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:09.974418 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65d5fc7-69ac-41f7-bc7d-a6565e658f3b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:09.976396 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65d5fc7-69ac-41f7-bc7d-a6565e658f3b None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:09.983803 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 103/206] 10.222.0.61 () {56 vars in 1374 bytes} [Mon Aug 22 19:05:09 2022] GET /identity/v3/role_assignments?user.id=14147cb34fc743e89b373b1a740217f8&scope.project.id=a4fe339e17324d15a22e51626818eecd&role.id=3f7ffde3fddd46caa6e49c24a6f07673 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:10.053677 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c928f9c-fb19-4f59-ab5a-10cb884e8975 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:10.094733 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78c758f-383a-4d0d-87d6-5c2befbde860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:10.095472 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78c758f-383a-4d0d-87d6-5c2befbde860 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:10.095784 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78c758f-383a-4d0d-87d6-5c2befbde860 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:10.096062 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78c758f-383a-4d0d-87d6-5c2befbde860 None admin] PATH_INFO: `/v3/projects/47adea686aa0431fb9766ac68f4bf165` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:10.097359 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78c758f-383a-4d0d-87d6-5c2befbde860 None admin] RBAC: Authorizing `identity:get_project(project_id=47adea686aa0431fb9766ac68f4bf165)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:10.098889 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78c758f-383a-4d0d-87d6-5c2befbde860 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:10.100615 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 104/207] 10.222.0.61 () {56 vars in 1186 bytes} [Mon Aug 22 19:05:09 2022] GET /identity/v3/projects/47adea686aa0431fb9766ac68f4bf165 => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:10.106777 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65d5fc7-69ac-41f7-bc7d-a6565e658f3b None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:10.232107 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65d5fc7-69ac-41f7-bc7d-a6565e658f3b None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:10.259972 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:10.268168 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9512621-dde1-405a-b19b-46164b8175c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:10.269109 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9512621-dde1-405a-b19b-46164b8175c5 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:10.269759 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9512621-dde1-405a-b19b-46164b8175c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:10.270033 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9512621-dde1-405a-b19b-46164b8175c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:10.270769 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9512621-dde1-405a-b19b-46164b8175c5 None admin] RBAC: Adding query filter params (role.id=9f0952608a704dfca5fdbe4046331f21, scope.project.id=47adea686aa0431fb9766ac68f4bf165, user.id=61d1f8624beb408cb3bd4cbbde27c9bc) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:10.271074 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9512621-dde1-405a-b19b-46164b8175c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:10.272562 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9512621-dde1-405a-b19b-46164b8175c5 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:10.278651 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 104/208] 10.222.0.61 () {56 vars in 1374 bytes} [Mon Aug 22 19:05:10 2022] GET /identity/v3/role_assignments?user.id=61d1f8624beb408cb3bd4cbbde27c9bc&scope.project.id=47adea686aa0431fb9766ac68f4bf165&role.id=9f0952608a704dfca5fdbe4046331f21 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:11.175014 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8ee10a-8437-492c-97ee-5998dbe3076f None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:11.175530 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8ee10a-8437-492c-97ee-5998dbe3076f None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:11.175674 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8ee10a-8437-492c-97ee-5998dbe3076f None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:11.177274 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 105/209] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:05:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:05:11.182586 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11808527-ea0e-4442-9141-149e6009a6c5 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:11.183388 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11808527-ea0e-4442-9141-149e6009a6c5 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:11.183886 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11808527-ea0e-4442-9141-149e6009a6c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:11.203360 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-11808527-ea0e-4442-9141-149e6009a6c5 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:05:11.204877 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-11808527-ea0e-4442-9141-149e6009a6c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:05:11.222570 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7193d8c-bd63-4068-954b-d9527187948d None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:11.222863 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7193d8c-bd63-4068-954b-d9527187948d None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:11.223135 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7193d8c-bd63-4068-954b-d9527187948d None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:11.224735 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 106/210] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:05:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:05:11.229776 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6502e199-9f97-4a19-b7cf-14adac294521 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:11.230062 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6502e199-9f97-4a19-b7cf-14adac294521 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:11.230342 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6502e199-9f97-4a19-b7cf-14adac294521 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:11.251261 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-6502e199-9f97-4a19-b7cf-14adac294521 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:05:11.252787 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-6502e199-9f97-4a19-b7cf-14adac294521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:05:11.279439 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 105/211] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:05:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 19:05:11.280036 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:11.325737 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 107/212] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:05:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 19:05:11.555502 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11808527-ea0e-4442-9141-149e6009a6c5 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:11.596085 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6502e199-9f97-4a19-b7cf-14adac294521 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:11.661703 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11808527-ea0e-4442-9141-149e6009a6c5 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:11.688109 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:11.699914 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6502e199-9f97-4a19-b7cf-14adac294521 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:11.700433 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991e5518-d527-495b-9b0d-fa512bec9d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:11.701415 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991e5518-d527-495b-9b0d-fa512bec9d1a None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:11.701914 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991e5518-d527-495b-9b0d-fa512bec9d1a None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:11.702437 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991e5518-d527-495b-9b0d-fa512bec9d1a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:11.713855 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991e5518-d527-495b-9b0d-fa512bec9d1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 19:05:11.720446 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-991e5518-d527-495b-9b0d-fa512bec9d1a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 22 19:05:11.720446 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:05:11.720446 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:05:11.720446 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:05:11.720446 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:05:11.720446 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:05:11.720446 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:05:11.720446 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:05:11.720446 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:05:11.720446 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:05:11.720446 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:05:11.720446 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:05:11.720446 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 19:05:11.720446 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 19:05:11.720446 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 19:05:11.720446 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 19:05:11.721324 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 19:05:11.721324 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 19:05:11.721324 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 19:05:11.721324 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 19:05:11.721324 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:05:11.721324 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:05:11.721324 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:05:11.721324 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:05:11.721324 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:05:11.721324 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:05:11.721324 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:05:11.721324 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:05:11.721324 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:05:11.721324 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:05:11.721324 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:05:11.721324 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:05:11.722113 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:05:11.722113 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:05:11.722113 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:05:11.722113 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:05:11.722113 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 19:05:11.722113 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 19:05:11.722113 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 19:05:11.722113 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 19:05:11.722113 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 19:05:11.722113 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 19:05:11.722113 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 22 19:05:11.722113 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:05:11.724579 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 106/213] 10.222.0.61 () {56 vars in 1099 bytes} [Mon Aug 22 19:05:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:05:11.730471 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991e5518-d527-495b-9b0d-fa512bec9d1a None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:11.736620 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c160812f-4651-429c-99ce-baa7515b7249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:11.736931 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c160812f-4651-429c-99ce-baa7515b7249 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:11.737210 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c160812f-4651-429c-99ce-baa7515b7249 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:11.737480 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c160812f-4651-429c-99ce-baa7515b7249 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:11.738869 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c160812f-4651-429c-99ce-baa7515b7249 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:11.740377 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c160812f-4651-429c-99ce-baa7515b7249 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:11.746736 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 108/214] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:05:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:11.747672 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c160812f-4651-429c-99ce-baa7515b7249 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:11.778335 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:11.845050 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991e5518-d527-495b-9b0d-fa512bec9d1a None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:11.853799 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c160812f-4651-429c-99ce-baa7515b7249 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:11.883543 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39967d47-c8cb-4dff-b683-b3eeb9e17720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:11.884557 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39967d47-c8cb-4dff-b683-b3eeb9e17720 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:11.885188 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39967d47-c8cb-4dff-b683-b3eeb9e17720 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:11.885778 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39967d47-c8cb-4dff-b683-b3eeb9e17720 None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:11.886658 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39967d47-c8cb-4dff-b683-b3eeb9e17720 None admin] RBAC: Adding query filter params (name=member) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:11.887156 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39967d47-c8cb-4dff-b683-b3eeb9e17720 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:11.889355 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39967d47-c8cb-4dff-b683-b3eeb9e17720 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:11.891107 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66433e7-c759-43fd-8aa1-8afa62adbd1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:11.891850 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66433e7-c759-43fd-8aa1-8afa62adbd1c None admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:11.892137 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66433e7-c759-43fd-8aa1-8afa62adbd1c None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:11.892415 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66433e7-c759-43fd-8aa1-8afa62adbd1c None admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:11.893064 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66433e7-c759-43fd-8aa1-8afa62adbd1c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:11.894620 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66433e7-c759-43fd-8aa1-8afa62adbd1c None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:11.900250 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 107/215] 10.222.0.61 () {56 vars in 1101 bytes} [Mon Aug 22 19:05:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 19:05:11.906532 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39967d47-c8cb-4dff-b683-b3eeb9e17720 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:11.927842 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 109/216] 10.222.0.61 () {60 vars in 1132 bytes} [Mon Aug 22 19:05:11 2022] POST /identity/v3/users => generated 297 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:05:11.960672 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:12.006747 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46320d0a-c730-43ee-b4c2-9ccf430db948 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:12.007060 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46320d0a-c730-43ee-b4c2-9ccf430db948 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:12.007306 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46320d0a-c730-43ee-b4c2-9ccf430db948 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:12.008774 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 110/217] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:05:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 19:05:12.014593 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13faead5-a2cf-4b8d-95df-cc328d8b15fb None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:12.014955 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13faead5-a2cf-4b8d-95df-cc328d8b15fb None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:12.015225 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13faead5-a2cf-4b8d-95df-cc328d8b15fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:12.025362 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39967d47-c8cb-4dff-b683-b3eeb9e17720 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:12.043628 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-13faead5-a2cf-4b8d-95df-cc328d8b15fb None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:05:12.045350 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-13faead5-a2cf-4b8d-95df-cc328d8b15fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:05:12.064291 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5299ab86-916c-464f-97e9-68029a62a1b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:12.064978 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5299ab86-916c-464f-97e9-68029a62a1b0 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:12.065256 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5299ab86-916c-464f-97e9-68029a62a1b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:12.065527 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5299ab86-916c-464f-97e9-68029a62a1b0 None admin] PATH_INFO: `/v3/users/17b0d9642212479abbf4a557562ba107` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:12.080625 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5299ab86-916c-464f-97e9-68029a62a1b0 None admin] RBAC: Authorizing `identity:get_user(user_id=17b0d9642212479abbf4a557562ba107)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:12.082104 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5299ab86-916c-464f-97e9-68029a62a1b0 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:12.083859 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 108/218] 10.222.0.61 () {56 vars in 1177 bytes} [Mon Aug 22 19:05:11 2022] GET /identity/v3/users/17b0d9642212479abbf4a557562ba107 => generated 297 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:12.089524 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5299ab86-916c-464f-97e9-68029a62a1b0 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:12.130117 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 111/219] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:05:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 19:05:12.212612 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5299ab86-916c-464f-97e9-68029a62a1b0 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:12.214451 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:12.235798 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1360e1e-5d42-40cf-9066-c772b16ca166 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:12.236302 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1360e1e-5d42-40cf-9066-c772b16ca166 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:12.236378 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1360e1e-5d42-40cf-9066-c772b16ca166 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:12.237732 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 112/220] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:05:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 19:05:12.243508 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3a58ce-4606-4d6d-9c33-b356273ece29 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:12.243872 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3a58ce-4606-4d6d-9c33-b356273ece29 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:12.244104 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3a58ce-4606-4d6d-9c33-b356273ece29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:12.254490 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf792375-84f2-4ccb-ae4b-ba460ef7f7fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:12.255112 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf792375-84f2-4ccb-ae4b-ba460ef7f7fa None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:12.255388 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf792375-84f2-4ccb-ae4b-ba460ef7f7fa None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:12.255690 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf792375-84f2-4ccb-ae4b-ba460ef7f7fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:12.265401 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-9e3a58ce-4606-4d6d-9c33-b356273ece29 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:05:12.266703 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-9e3a58ce-4606-4d6d-9c33-b356273ece29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:05:12.279556 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:12.346263 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf792375-84f2-4ccb-ae4b-ba460ef7f7fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:12.347935 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf792375-84f2-4ccb-ae4b-ba460ef7f7fa None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:12.358724 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 113/221] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:05:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 19:05:12.431525 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3a58ce-4606-4d6d-9c33-b356273ece29 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:12.458449 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 109/222] 10.222.0.61 () {58 vars in 1282 bytes} [Mon Aug 22 19:05:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 19:05:12.469271 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf792375-84f2-4ccb-ae4b-ba460ef7f7fa None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:12.471287 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:12.555137 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3a58ce-4606-4d6d-9c33-b356273ece29 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:12.604367 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ee56d2-4e7f-4828-a95f-94b43458d2ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:12.605025 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ee56d2-4e7f-4828-a95f-94b43458d2ff None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:12.605306 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ee56d2-4e7f-4828-a95f-94b43458d2ff None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:12.605579 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ee56d2-4e7f-4828-a95f-94b43458d2ff None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:12.621047 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ee56d2-4e7f-4828-a95f-94b43458d2ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 19:05:12.621669 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf792375-84f2-4ccb-ae4b-ba460ef7f7fa None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:12.626038 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-35ee56d2-4e7f-4828-a95f-94b43458d2ff None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 19:05:12.626038 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:05:12.626038 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:05:12.626038 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:05:12.626038 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:05:12.626038 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:05:12.626038 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:05:12.626038 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:05:12.626038 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:05:12.626038 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:05:12.626038 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:05:12.626038 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:05:12.626038 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 19:05:12.626038 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 19:05:12.626038 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 19:05:12.626038 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 19:05:12.626935 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 19:05:12.626935 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 19:05:12.626935 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 19:05:12.626935 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 19:05:12.626935 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:05:12.626935 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:05:12.626935 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:05:12.626935 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:05:12.626935 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:05:12.626935 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:05:12.626935 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:05:12.626935 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:05:12.626935 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:05:12.626935 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:05:12.626935 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:05:12.626935 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:05:12.627776 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:05:12.627776 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:05:12.627776 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:05:12.627776 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:05:12.627776 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 19:05:12.627776 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 19:05:12.627776 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 19:05:12.627776 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 19:05:12.627776 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 19:05:12.627776 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 19:05:12.627776 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 19:05:12.627776 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:05:12.627776 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 114/223] 10.222.0.61 () {56 vars in 1096 bytes} [Mon Aug 22 19:05:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 19:05:12.637366 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ee56d2-4e7f-4828-a95f-94b43458d2ff None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:12.655799 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:12.672801 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2528ecfe-eaaa-41b5-a429-e30ad797709f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:12.673444 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2528ecfe-eaaa-41b5-a429-e30ad797709f None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:12.673716 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2528ecfe-eaaa-41b5-a429-e30ad797709f None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:12.673984 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2528ecfe-eaaa-41b5-a429-e30ad797709f None admin] PATH_INFO: `/v3/projects/47adea686aa0431fb9766ac68f4bf165` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:12.675119 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2528ecfe-eaaa-41b5-a429-e30ad797709f None admin] RBAC: Authorizing `identity:get_project(project_id=47adea686aa0431fb9766ac68f4bf165)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:12.676632 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2528ecfe-eaaa-41b5-a429-e30ad797709f None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:12.678450 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 110/224] 10.222.0.61 () {56 vars in 1186 bytes} [Mon Aug 22 19:05:12 2022] GET /identity/v3/projects/47adea686aa0431fb9766ac68f4bf165 => generated 302 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:12.679117 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2528ecfe-eaaa-41b5-a429-e30ad797709f None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:12.763011 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:12.779796 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ee56d2-4e7f-4828-a95f-94b43458d2ff None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:12.801358 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2528ecfe-eaaa-41b5-a429-e30ad797709f None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:12.827312 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5fa515-6544-4eae-a77e-4be596db0601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:12.828695 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5fa515-6544-4eae-a77e-4be596db0601 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:12.829347 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5fa515-6544-4eae-a77e-4be596db0601 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:12.830162 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5fa515-6544-4eae-a77e-4be596db0601 None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:12.831244 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5fa515-6544-4eae-a77e-4be596db0601 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:12.831702 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5fa515-6544-4eae-a77e-4be596db0601 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:12.834064 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5fa515-6544-4eae-a77e-4be596db0601 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:12.837458 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9019e2-bd16-4591-beda-82ba21a71264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:12.838406 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9019e2-bd16-4591-beda-82ba21a71264 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:12.838684 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9019e2-bd16-4591-beda-82ba21a71264 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:12.838955 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9019e2-bd16-4591-beda-82ba21a71264 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:12.845099 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 115/225] 10.222.0.61 () {56 vars in 1099 bytes} [Mon Aug 22 19:05:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:12.846167 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5fa515-6544-4eae-a77e-4be596db0601 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:12.852128 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9019e2-bd16-4591-beda-82ba21a71264 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 19:05:12.857081 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-ba9019e2-bd16-4591-beda-82ba21a71264 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 19:05:12.857081 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:05:12.857081 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:05:12.857081 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:05:12.857081 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:05:12.857081 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:05:12.857081 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:05:12.857081 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:05:12.857081 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:05:12.857081 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:05:12.857081 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:05:12.857081 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:05:12.857081 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 19:05:12.857081 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 19:05:12.857081 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 19:05:12.857081 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 19:05:12.857992 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 19:05:12.857992 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 19:05:12.857992 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 19:05:12.857992 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 19:05:12.857992 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:05:12.857992 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:05:12.857992 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:05:12.857992 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:05:12.857992 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:05:12.857992 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:05:12.857992 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:05:12.857992 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:05:12.857992 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:05:12.857992 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:05:12.857992 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:05:12.857992 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:05:12.858780 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:05:12.858780 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:05:12.858780 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:05:12.858780 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:05:12.858780 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 19:05:12.858780 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 19:05:12.858780 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 19:05:12.858780 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 19:05:12.858780 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 19:05:12.858780 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 19:05:12.858780 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 19:05:12.858780 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:05:12.858780 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 111/226] 10.222.0.61 () {56 vars in 1114 bytes} [Mon Aug 22 19:05:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:05:12.859466 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9019e2-bd16-4591-beda-82ba21a71264 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:12.947669 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:12.969531 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9019e2-bd16-4591-beda-82ba21a71264 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:12.997163 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5fa515-6544-4eae-a77e-4be596db0601 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:13.002326 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:13.008644 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40936946-54eb-4189-81d9-eacbff7c1fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:13.009286 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40936946-54eb-4189-81d9-eacbff7c1fdb None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:13.009728 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40936946-54eb-4189-81d9-eacbff7c1fdb None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:13.010459 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40936946-54eb-4189-81d9-eacbff7c1fdb None admin] PATH_INFO: `/v3/users/61d1f8624beb408cb3bd4cbbde27c9bc` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:13.012156 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40936946-54eb-4189-81d9-eacbff7c1fdb None admin] RBAC: Authorizing `identity:get_user(user_id=61d1f8624beb408cb3bd4cbbde27c9bc)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:13.013800 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40936946-54eb-4189-81d9-eacbff7c1fdb None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:13.016359 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 112/227] 10.222.0.61 () {56 vars in 1177 bytes} [Mon Aug 22 19:05:12 2022] GET /identity/v3/users/61d1f8624beb408cb3bd4cbbde27c9bc => generated 283 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:13.017276 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40936946-54eb-4189-81d9-eacbff7c1fdb None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:13.044457 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09883fa-029e-4525-a676-58f455f80a7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:13.045065 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09883fa-029e-4525-a676-58f455f80a7b None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:13.045339 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09883fa-029e-4525-a676-58f455f80a7b None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:13.045607 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09883fa-029e-4525-a676-58f455f80a7b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:13.046332 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09883fa-029e-4525-a676-58f455f80a7b None admin] RBAC: Adding query filter params (role.id=74a40bc819cb42eea7e966a4b2a016f1, scope.project.id=47adea686aa0431fb9766ac68f4bf165, user.id=17b0d9642212479abbf4a557562ba107) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:13.046636 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09883fa-029e-4525-a676-58f455f80a7b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:13.048132 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09883fa-029e-4525-a676-58f455f80a7b None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:13.053752 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 116/228] 10.222.0.61 () {56 vars in 1374 bytes} [Mon Aug 22 19:05:12 2022] GET /identity/v3/role_assignments?user.id=17b0d9642212479abbf4a557562ba107&scope.project.id=47adea686aa0431fb9766ac68f4bf165&role.id=74a40bc819cb42eea7e966a4b2a016f1 => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:13.054596 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09883fa-029e-4525-a676-58f455f80a7b None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:13.174860 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09883fa-029e-4525-a676-58f455f80a7b None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:13.197508 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40936946-54eb-4189-81d9-eacbff7c1fdb None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:13.210428 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:13.221165 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99982b08-69be-476d-b8c5-c2d5e641e1fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:13.221822 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99982b08-69be-476d-b8c5-c2d5e641e1fc None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:13.222097 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99982b08-69be-476d-b8c5-c2d5e641e1fc None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:13.222367 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99982b08-69be-476d-b8c5-c2d5e641e1fc None admin] PATH_INFO: `/v3/projects/47adea686aa0431fb9766ac68f4bf165` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:13.225710 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99982b08-69be-476d-b8c5-c2d5e641e1fc None admin] RBAC: Authorizing `identity:get_project(project_id=47adea686aa0431fb9766ac68f4bf165)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:13.228734 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99982b08-69be-476d-b8c5-c2d5e641e1fc None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:13.230537 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 117/229] 10.222.0.61 () {56 vars in 1186 bytes} [Mon Aug 22 19:05:13 2022] GET /identity/v3/projects/47adea686aa0431fb9766ac68f4bf165 => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:13.236881 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99982b08-69be-476d-b8c5-c2d5e641e1fc None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:13.277925 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f734acf-4432-41df-b1e4-5d61d59273a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:13.278561 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f734acf-4432-41df-b1e4-5d61d59273a6 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:13.278794 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f734acf-4432-41df-b1e4-5d61d59273a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:13.279079 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f734acf-4432-41df-b1e4-5d61d59273a6 None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:13.280059 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f734acf-4432-41df-b1e4-5d61d59273a6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:13.280357 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f734acf-4432-41df-b1e4-5d61d59273a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:13.282581 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f734acf-4432-41df-b1e4-5d61d59273a6 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:13.294433 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 113/230] 10.222.0.61 () {56 vars in 1111 bytes} [Mon Aug 22 19:05:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:13.300444 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f734acf-4432-41df-b1e4-5d61d59273a6 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:13.385900 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99982b08-69be-476d-b8c5-c2d5e641e1fc None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:13.436496 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:13.459965 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f21069d-1708-4c83-b3b2-9579028d3189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:13.460642 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f21069d-1708-4c83-b3b2-9579028d3189 None admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:13.460922 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f21069d-1708-4c83-b3b2-9579028d3189 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:13.461190 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f21069d-1708-4c83-b3b2-9579028d3189 None admin] PATH_INFO: `/v3/projects/47adea686aa0431fb9766ac68f4bf165/users/61d1f8624beb408cb3bd4cbbde27c9bc/roles/163c30c3b3c246beb849ce176f02d9da` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:13.467724 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f21069d-1708-4c83-b3b2-9579028d3189 None admin] RBAC: Authorizing `identity:create_grant(project_id=47adea686aa0431fb9766ac68f4bf165, user_id=61d1f8624beb408cb3bd4cbbde27c9bc, role_id=163c30c3b3c246beb849ce176f02d9da)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:13.469238 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f21069d-1708-4c83-b3b2-9579028d3189 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:13.495127 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 118/231] 10.222.0.61 () {58 vars in 1439 bytes} [Mon Aug 22 19:05:13 2022] PUT /identity/v3/projects/47adea686aa0431fb9766ac68f4bf165/users/61d1f8624beb408cb3bd4cbbde27c9bc/roles/163c30c3b3c246beb849ce176f02d9da => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:05:13.513892 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f734acf-4432-41df-b1e4-5d61d59273a6 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:13.565795 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce74072d-b88e-416d-b900-9700e343c779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:13.566636 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce74072d-b88e-416d-b900-9700e343c779 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:13.566916 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce74072d-b88e-416d-b900-9700e343c779 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:13.567190 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce74072d-b88e-416d-b900-9700e343c779 None admin] PATH_INFO: `/v3/users/61d1f8624beb408cb3bd4cbbde27c9bc` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:13.568588 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce74072d-b88e-416d-b900-9700e343c779 None admin] RBAC: Authorizing `identity:get_user(user_id=61d1f8624beb408cb3bd4cbbde27c9bc)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:13.570114 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce74072d-b88e-416d-b900-9700e343c779 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:13.574478 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 114/232] 10.222.0.61 () {56 vars in 1177 bytes} [Mon Aug 22 19:05:13 2022] GET /identity/v3/users/61d1f8624beb408cb3bd4cbbde27c9bc => generated 283 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 19:05:13.580114 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f21069d-1708-4c83-b3b2-9579028d3189 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:13.664008 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:13.713226 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f21069d-1708-4c83-b3b2-9579028d3189 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:13.749487 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb03946-da33-4891-9ac4-70184754680e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:13.750163 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb03946-da33-4891-9ac4-70184754680e None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:13.750441 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb03946-da33-4891-9ac4-70184754680e None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:13.750714 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb03946-da33-4891-9ac4-70184754680e None admin] PATH_INFO: `/v3/projects/47adea686aa0431fb9766ac68f4bf165` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:13.751923 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb03946-da33-4891-9ac4-70184754680e None admin] RBAC: Authorizing `identity:get_project(project_id=47adea686aa0431fb9766ac68f4bf165)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:13.753380 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb03946-da33-4891-9ac4-70184754680e None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:13.755239 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 119/233] 10.222.0.61 () {56 vars in 1186 bytes} [Mon Aug 22 19:05:13 2022] GET /identity/v3/projects/47adea686aa0431fb9766ac68f4bf165 => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:13.760877 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce74072d-b88e-416d-b900-9700e343c779 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:13.872138 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce74072d-b88e-416d-b900-9700e343c779 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:13.899038 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:13.909221 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a483a62-e781-4acd-8a8e-2578129ba3cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:13.909944 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a483a62-e781-4acd-8a8e-2578129ba3cd None admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:13.910222 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a483a62-e781-4acd-8a8e-2578129ba3cd None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:13.910494 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a483a62-e781-4acd-8a8e-2578129ba3cd None admin] PATH_INFO: `/v3/projects/47adea686aa0431fb9766ac68f4bf165/users/61d1f8624beb408cb3bd4cbbde27c9bc/roles/9f0952608a704dfca5fdbe4046331f21` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:13.912712 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a483a62-e781-4acd-8a8e-2578129ba3cd None admin] RBAC: Authorizing `identity:create_grant(project_id=47adea686aa0431fb9766ac68f4bf165, user_id=61d1f8624beb408cb3bd4cbbde27c9bc, role_id=9f0952608a704dfca5fdbe4046331f21)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:13.914149 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a483a62-e781-4acd-8a8e-2578129ba3cd None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:13.930172 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 115/234] 10.222.0.61 () {58 vars in 1439 bytes} [Mon Aug 22 19:05:13 2022] PUT /identity/v3/projects/47adea686aa0431fb9766ac68f4bf165/users/61d1f8624beb408cb3bd4cbbde27c9bc/roles/9f0952608a704dfca5fdbe4046331f21 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:05:14.423073 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f03996-b265-4ffe-913d-110ebc7564b0 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:14.423587 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f03996-b265-4ffe-913d-110ebc7564b0 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:14.423587 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f03996-b265-4ffe-913d-110ebc7564b0 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:14.424955 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 120/235] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:05:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:05:14.430388 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9a0b9a-827b-4a1e-962b-79e275e008f6 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:14.430885 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9a0b9a-827b-4a1e-962b-79e275e008f6 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:14.431051 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9a0b9a-827b-4a1e-962b-79e275e008f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:14.454139 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-fd9a0b9a-827b-4a1e-962b-79e275e008f6 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:05:14.455326 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-fd9a0b9a-827b-4a1e-962b-79e275e008f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:05:14.467882 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe207329-a540-4a77-a8ff-e8c6d1eceb48 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:14.468162 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe207329-a540-4a77-a8ff-e8c6d1eceb48 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:14.468433 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe207329-a540-4a77-a8ff-e8c6d1eceb48 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:14.470718 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 121/236] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:05:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 19:05:14.475697 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb6835a-8500-4b19-a623-77615a6e6cb7 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:14.475979 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb6835a-8500-4b19-a623-77615a6e6cb7 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:14.476251 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb6835a-8500-4b19-a623-77615a6e6cb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:14.508841 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-2fb6835a-8500-4b19-a623-77615a6e6cb7 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:05:14.510033 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-2fb6835a-8500-4b19-a623-77615a6e6cb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:05:14.601115 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 116/237] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:05:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 19:05:14.611097 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 122/238] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:05:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 19:05:14.924655 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9a0b9a-827b-4a1e-962b-79e275e008f6 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:14.987163 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb6835a-8500-4b19-a623-77615a6e6cb7 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:15.058993 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:15.071573 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9a0b9a-827b-4a1e-962b-79e275e008f6 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:15.111145 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb6835a-8500-4b19-a623-77615a6e6cb7 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:15.122639 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:15.154263 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b719e7c6-5962-4940-bd75-9492e5820a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:15.154907 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b719e7c6-5962-4940-bd75-9492e5820a4c None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:15.155181 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b719e7c6-5962-4940-bd75-9492e5820a4c None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:15.155447 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b719e7c6-5962-4940-bd75-9492e5820a4c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:15.156741 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b719e7c6-5962-4940-bd75-9492e5820a4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:15.158622 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b719e7c6-5962-4940-bd75-9492e5820a4c None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:15.160387 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 123/239] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:05:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 19:05:15.165859 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b719e7c6-5962-4940-bd75-9492e5820a4c None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:15.166536 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5cb171-33ef-4343-a75e-5e0f77700c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:15.167399 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5cb171-33ef-4343-a75e-5e0f77700c81 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:15.167932 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5cb171-33ef-4343-a75e-5e0f77700c81 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:15.168403 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5cb171-33ef-4343-a75e-5e0f77700c81 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:15.200410 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5cb171-33ef-4343-a75e-5e0f77700c81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 19:05:15.205662 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-4e5cb171-33ef-4343-a75e-5e0f77700c81 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 22 19:05:15.205662 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:05:15.205662 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:05:15.205662 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:05:15.205662 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:05:15.205662 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:05:15.205662 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:05:15.205662 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:05:15.205662 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:05:15.205662 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:05:15.205662 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:05:15.205662 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:05:15.205662 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 19:05:15.205662 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 19:05:15.205662 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 19:05:15.205662 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 19:05:15.206544 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 19:05:15.206544 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 19:05:15.206544 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 19:05:15.206544 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 19:05:15.206544 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:05:15.206544 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:05:15.206544 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:05:15.206544 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:05:15.206544 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:05:15.206544 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:05:15.206544 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:05:15.206544 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:05:15.206544 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:05:15.206544 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:05:15.206544 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:05:15.206544 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:05:15.207332 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:05:15.207332 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:05:15.207332 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:05:15.207332 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:05:15.207332 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 19:05:15.207332 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 19:05:15.207332 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 19:05:15.207332 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 19:05:15.207332 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 19:05:15.207332 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 19:05:15.207332 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 22 19:05:15.207332 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:05:15.209960 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 117/240] 10.222.0.61 () {56 vars in 1099 bytes} [Mon Aug 22 19:05:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:05:15.215750 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5cb171-33ef-4343-a75e-5e0f77700c81 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:15.282574 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b719e7c6-5962-4940-bd75-9492e5820a4c None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:15.310067 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:15.320718 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7cd9aa-4bef-4348-a572-b4b013140ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:15.321330 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7cd9aa-4bef-4348-a572-b4b013140ccf None admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:15.321604 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7cd9aa-4bef-4348-a572-b4b013140ccf None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:15.321875 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7cd9aa-4bef-4348-a572-b4b013140ccf None admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:15.322515 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7cd9aa-4bef-4348-a572-b4b013140ccf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:15.324161 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7cd9aa-4bef-4348-a572-b4b013140ccf None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:15.352699 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 124/241] 10.222.0.61 () {60 vars in 1132 bytes} [Mon Aug 22 19:05:15 2022] POST /identity/v3/users => generated 293 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 19:05:15.367281 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5cb171-33ef-4343-a75e-5e0f77700c81 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:15.379774 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abae1a28-398b-4e16-ae7b-32d4793f3901 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:15.380398 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abae1a28-398b-4e16-ae7b-32d4793f3901 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:15.381272 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abae1a28-398b-4e16-ae7b-32d4793f3901 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:15.383090 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 125/242] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:05:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:05:15.388997 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c22724-29f6-41b3-9207-e241f04227d0 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:15.389555 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c22724-29f6-41b3-9207-e241f04227d0 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:15.390030 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c22724-29f6-41b3-9207-e241f04227d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:15.414919 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-10c22724-29f6-41b3-9207-e241f04227d0 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:05:15.417860 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-10c22724-29f6-41b3-9207-e241f04227d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:05:15.440912 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a037d8-1db8-47aa-9e3c-a93dda1718b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:15.442247 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a037d8-1db8-47aa-9e3c-a93dda1718b3 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:15.442725 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a037d8-1db8-47aa-9e3c-a93dda1718b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:15.443193 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a037d8-1db8-47aa-9e3c-a93dda1718b3 None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:15.444177 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a037d8-1db8-47aa-9e3c-a93dda1718b3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:15.444701 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a037d8-1db8-47aa-9e3c-a93dda1718b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:15.446309 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a037d8-1db8-47aa-9e3c-a93dda1718b3 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:15.461387 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 118/243] 10.222.0.61 () {56 vars in 1101 bytes} [Mon Aug 22 19:05:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:15.469246 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a037d8-1db8-47aa-9e3c-a93dda1718b3 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:15.539734 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 126/244] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:05:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 19:05:15.545742 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:15.610078 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a037d8-1db8-47aa-9e3c-a93dda1718b3 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:15.645343 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84352fc2-4e5f-4da6-90f3-68847c4812cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:15.646372 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84352fc2-4e5f-4da6-90f3-68847c4812cf None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:15.646853 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84352fc2-4e5f-4da6-90f3-68847c4812cf None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:15.647339 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84352fc2-4e5f-4da6-90f3-68847c4812cf None admin] PATH_INFO: `/v3/users/e06c4ef6c379460bba9b1190a073b191` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:15.662039 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84352fc2-4e5f-4da6-90f3-68847c4812cf None admin] RBAC: Authorizing `identity:get_user(user_id=e06c4ef6c379460bba9b1190a073b191)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:15.663836 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84352fc2-4e5f-4da6-90f3-68847c4812cf None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:15.666027 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 119/245] 10.222.0.61 () {56 vars in 1177 bytes} [Mon Aug 22 19:05:15 2022] GET /identity/v3/users/e06c4ef6c379460bba9b1190a073b191 => generated 297 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:15.671700 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c22724-29f6-41b3-9207-e241f04227d0 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:15.800407 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c22724-29f6-41b3-9207-e241f04227d0 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:15.818619 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:15.846469 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea0f474-e34a-4680-9f89-2dd80709f497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:15.847426 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea0f474-e34a-4680-9f89-2dd80709f497 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:15.847925 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea0f474-e34a-4680-9f89-2dd80709f497 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:15.848390 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea0f474-e34a-4680-9f89-2dd80709f497 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:15.867114 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84352fc2-4e5f-4da6-90f3-68847c4812cf None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:15.926641 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea0f474-e34a-4680-9f89-2dd80709f497 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:15.928452 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea0f474-e34a-4680-9f89-2dd80709f497 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:15.992968 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:16.002463 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84352fc2-4e5f-4da6-90f3-68847c4812cf None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:16.048276 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f283e586-99a7-453c-804a-4a14f72e1870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:16.048917 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f283e586-99a7-453c-804a-4a14f72e1870 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:16.049190 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f283e586-99a7-453c-804a-4a14f72e1870 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:16.049458 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f283e586-99a7-453c-804a-4a14f72e1870 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:16.051803 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 127/246] 10.222.0.61 () {58 vars in 1282 bytes} [Mon Aug 22 19:05:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 19:05:16.057583 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea0f474-e34a-4680-9f89-2dd80709f497 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:16.068604 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f283e586-99a7-453c-804a-4a14f72e1870 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 19:05:16.074083 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:16.076644 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-f283e586-99a7-453c-804a-4a14f72e1870 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 22 19:05:16.076644 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:05:16.076644 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:05:16.076644 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:05:16.076644 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:05:16.076644 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:05:16.076644 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:05:16.076644 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:05:16.076644 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:05:16.076644 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:05:16.076644 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:05:16.076644 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:05:16.076644 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 19:05:16.076644 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 19:05:16.076644 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 19:05:16.076644 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 19:05:16.077782 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 19:05:16.077782 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 19:05:16.077782 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 19:05:16.077782 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 19:05:16.077782 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:05:16.077782 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:05:16.077782 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:05:16.077782 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:05:16.077782 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:05:16.077782 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:05:16.077782 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:05:16.077782 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:05:16.077782 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:05:16.077782 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:05:16.077782 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:05:16.077782 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:05:16.079226 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:05:16.079226 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:05:16.079226 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:05:16.079226 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:05:16.079226 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 19:05:16.079226 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 19:05:16.079226 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 19:05:16.079226 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 19:05:16.079226 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 19:05:16.079226 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 19:05:16.079226 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 22 19:05:16.079226 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:05:16.079226 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 120/247] 10.222.0.61 () {56 vars in 1099 bytes} [Mon Aug 22 19:05:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:05:16.084724 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f283e586-99a7-453c-804a-4a14f72e1870 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:16.195225 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea0f474-e34a-4680-9f89-2dd80709f497 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:16.224509 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f283e586-99a7-453c-804a-4a14f72e1870 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:16.235776 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02e34e5-3fad-4ac6-b7ca-e1e9bd3f7cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:16.236769 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02e34e5-3fad-4ac6-b7ca-e1e9bd3f7cda None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:16.237248 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02e34e5-3fad-4ac6-b7ca-e1e9bd3f7cda None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:16.237717 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02e34e5-3fad-4ac6-b7ca-e1e9bd3f7cda None admin] PATH_INFO: `/v3/projects/47adea686aa0431fb9766ac68f4bf165` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:16.239355 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02e34e5-3fad-4ac6-b7ca-e1e9bd3f7cda None admin] RBAC: Authorizing `identity:get_project(project_id=47adea686aa0431fb9766ac68f4bf165)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:16.241046 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02e34e5-3fad-4ac6-b7ca-e1e9bd3f7cda None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:16.243244 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 128/248] 10.222.0.61 () {56 vars in 1186 bytes} [Mon Aug 22 19:05:16 2022] GET /identity/v3/projects/47adea686aa0431fb9766ac68f4bf165 => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 19:05:16.244706 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cb04ad-6a36-4444-b6b4-9bd993cba6b4 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:16.245192 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cb04ad-6a36-4444-b6b4-9bd993cba6b4 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:16.245693 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cb04ad-6a36-4444-b6b4-9bd993cba6b4 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:16.247374 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 129/249] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:05:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:05:16.252135 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cb04ad-6a36-4444-b6b4-9bd993cba6b4 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:16.274109 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d278f1-4b03-4d7c-a282-91fe0c848688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:16.274821 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d278f1-4b03-4d7c-a282-91fe0c848688 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:16.275558 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d278f1-4b03-4d7c-a282-91fe0c848688 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:16.275558 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d278f1-4b03-4d7c-a282-91fe0c848688 None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:16.276328 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d278f1-4b03-4d7c-a282-91fe0c848688 None admin] RBAC: Adding query filter params (name=member) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:16.276690 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d278f1-4b03-4d7c-a282-91fe0c848688 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:16.278128 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d278f1-4b03-4d7c-a282-91fe0c848688 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:16.287572 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 121/250] 10.222.0.61 () {56 vars in 1101 bytes} [Mon Aug 22 19:05:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:16.288867 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b29a0f-7e4a-4c95-a66e-c58699864264 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:16.289222 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b29a0f-7e4a-4c95-a66e-c58699864264 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:16.289563 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b29a0f-7e4a-4c95-a66e-c58699864264 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:16.294539 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 122/251] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:05:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:05:16.295836 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1b5116-77af-4574-8a23-227e27bf29f7 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:16.296201 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1b5116-77af-4574-8a23-227e27bf29f7 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:16.296549 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1b5116-77af-4574-8a23-227e27bf29f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:16.315888 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-1e1b5116-77af-4574-8a23-227e27bf29f7 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:05:16.317191 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-1e1b5116-77af-4574-8a23-227e27bf29f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:05:16.333689 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:16.346634 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:16.368209 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cb04ad-6a36-4444-b6b4-9bd993cba6b4 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:16.403670 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 123/252] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:05:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 19:05:16.403670 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1b5116-77af-4574-8a23-227e27bf29f7 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:16.417686 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478ff439-7ea4-4ad8-b1a7-dea5a7f58d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:16.418568 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478ff439-7ea4-4ad8-b1a7-dea5a7f58d9c None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:16.419016 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478ff439-7ea4-4ad8-b1a7-dea5a7f58d9c None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:16.419569 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478ff439-7ea4-4ad8-b1a7-dea5a7f58d9c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:16.420499 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478ff439-7ea4-4ad8-b1a7-dea5a7f58d9c None admin] RBAC: Adding query filter params (role.id=3f7ffde3fddd46caa6e49c24a6f07673, scope.project.id=47adea686aa0431fb9766ac68f4bf165, user.id=e06c4ef6c379460bba9b1190a073b191) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:16.420988 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478ff439-7ea4-4ad8-b1a7-dea5a7f58d9c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:16.423151 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478ff439-7ea4-4ad8-b1a7-dea5a7f58d9c None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:16.434702 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 130/253] 10.222.0.61 () {56 vars in 1374 bytes} [Mon Aug 22 19:05:16 2022] GET /identity/v3/role_assignments?user.id=e06c4ef6c379460bba9b1190a073b191&scope.project.id=47adea686aa0431fb9766ac68f4bf165&role.id=3f7ffde3fddd46caa6e49c24a6f07673 => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:16.436258 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8145a085-4d22-42d2-8457-7bbba4d69dc3 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:16.436724 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8145a085-4d22-42d2-8457-7bbba4d69dc3 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:16.437178 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8145a085-4d22-42d2-8457-7bbba4d69dc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:16.472798 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-8145a085-4d22-42d2-8457-7bbba4d69dc3 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:05:16.474367 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-8145a085-4d22-42d2-8457-7bbba4d69dc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:05:16.510589 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1b5116-77af-4574-8a23-227e27bf29f7 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:16.550054 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12f957a-5b2c-42b4-8a2a-f5db22d79f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:16.551278 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12f957a-5b2c-42b4-8a2a-f5db22d79f92 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:16.551278 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12f957a-5b2c-42b4-8a2a-f5db22d79f92 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:16.551506 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12f957a-5b2c-42b4-8a2a-f5db22d79f92 None admin] PATH_INFO: `/v3/users/17b0d9642212479abbf4a557562ba107` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:16.552883 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12f957a-5b2c-42b4-8a2a-f5db22d79f92 None admin] RBAC: Authorizing `identity:get_user(user_id=17b0d9642212479abbf4a557562ba107)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:16.554587 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12f957a-5b2c-42b4-8a2a-f5db22d79f92 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:16.556551 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 124/254] 10.222.0.61 () {56 vars in 1177 bytes} [Mon Aug 22 19:05:16 2022] GET /identity/v3/users/17b0d9642212479abbf4a557562ba107 => generated 297 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:16.565410 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12f957a-5b2c-42b4-8a2a-f5db22d79f92 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:16.584736 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 131/255] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:05:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 19:05:16.611228 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:16.713300 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12f957a-5b2c-42b4-8a2a-f5db22d79f92 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:16.758399 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e4125f-d364-47db-b720-9b43ac8c8a7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:16.759281 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e4125f-d364-47db-b720-9b43ac8c8a7b None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:16.759727 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e4125f-d364-47db-b720-9b43ac8c8a7b None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:16.760205 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e4125f-d364-47db-b720-9b43ac8c8a7b None admin] PATH_INFO: `/v3/projects/47adea686aa0431fb9766ac68f4bf165` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:16.764351 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e4125f-d364-47db-b720-9b43ac8c8a7b None admin] RBAC: Authorizing `identity:get_project(project_id=47adea686aa0431fb9766ac68f4bf165)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:16.766093 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e4125f-d364-47db-b720-9b43ac8c8a7b None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:16.767940 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 125/256] 10.222.0.61 () {56 vars in 1186 bytes} [Mon Aug 22 19:05:16 2022] GET /identity/v3/projects/47adea686aa0431fb9766ac68f4bf165 => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:16.773773 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8145a085-4d22-42d2-8457-7bbba4d69dc3 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:16.855136 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:16.881501 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8145a085-4d22-42d2-8457-7bbba4d69dc3 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:16.909659 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e4125f-d364-47db-b720-9b43ac8c8a7b None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:16.926006 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135a837e-4c34-47aa-9059-4d95549d1239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:16.926907 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135a837e-4c34-47aa-9059-4d95549d1239 None admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:16.927354 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135a837e-4c34-47aa-9059-4d95549d1239 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:16.927966 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135a837e-4c34-47aa-9059-4d95549d1239 None admin] PATH_INFO: `/v3/projects/47adea686aa0431fb9766ac68f4bf165/users/17b0d9642212479abbf4a557562ba107/roles/74a40bc819cb42eea7e966a4b2a016f1` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:16.931925 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135a837e-4c34-47aa-9059-4d95549d1239 None admin] RBAC: Authorizing `identity:create_grant(project_id=47adea686aa0431fb9766ac68f4bf165, user_id=17b0d9642212479abbf4a557562ba107, role_id=74a40bc819cb42eea7e966a4b2a016f1)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:16.934675 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135a837e-4c34-47aa-9059-4d95549d1239 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:16.958600 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 132/257] 10.222.0.61 () {58 vars in 1439 bytes} [Mon Aug 22 19:05:16 2022] PUT /identity/v3/projects/47adea686aa0431fb9766ac68f4bf165/users/17b0d9642212479abbf4a557562ba107/roles/74a40bc819cb42eea7e966a4b2a016f1 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 19:05:16.960068 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135a837e-4c34-47aa-9059-4d95549d1239 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:17.092677 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135a837e-4c34-47aa-9059-4d95549d1239 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:17.128273 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:17.132834 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5995eda2-59e4-4ccd-be4c-f9d8426d2989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:17.133854 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5995eda2-59e4-4ccd-be4c-f9d8426d2989 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:17.134305 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5995eda2-59e4-4ccd-be4c-f9d8426d2989 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:17.134902 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5995eda2-59e4-4ccd-be4c-f9d8426d2989 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:17.137261 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:17.150639 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5995eda2-59e4-4ccd-be4c-f9d8426d2989 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 19:05:17.157268 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-5995eda2-59e4-4ccd-be4c-f9d8426d2989 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 19:05:17.157268 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:05:17.157268 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:05:17.157268 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:05:17.157268 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:05:17.157268 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:05:17.157268 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:05:17.157268 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:05:17.157268 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:05:17.157268 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:05:17.157268 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:05:17.157268 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:05:17.157268 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 19:05:17.157268 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 19:05:17.157268 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 19:05:17.157268 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 19:05:17.158301 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 19:05:17.158301 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 19:05:17.158301 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 19:05:17.158301 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 19:05:17.158301 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:05:17.158301 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:05:17.158301 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:05:17.158301 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:05:17.158301 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:05:17.158301 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:05:17.158301 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:05:17.158301 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:05:17.158301 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:05:17.158301 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:05:17.158301 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:05:17.158301 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:05:17.159126 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:05:17.159126 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:05:17.159126 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:05:17.159126 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:05:17.159126 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 19:05:17.159126 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 19:05:17.159126 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 19:05:17.159126 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 19:05:17.159126 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 19:05:17.159126 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 19:05:17.159126 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 19:05:17.159126 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:05:17.161424 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 133/258] 10.222.0.61 () {56 vars in 1114 bytes} [Mon Aug 22 19:05:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:05:17.167560 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e4125f-d364-47db-b720-9b43ac8c8a7b None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:17.172851 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5995eda2-59e4-4ccd-be4c-f9d8426d2989 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:17.228683 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15424ed9-3be4-4cc8-99a0-eceba823ac5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:17.229676 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15424ed9-3be4-4cc8-99a0-eceba823ac5c None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:17.230098 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15424ed9-3be4-4cc8-99a0-eceba823ac5c None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:17.230513 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15424ed9-3be4-4cc8-99a0-eceba823ac5c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:17.249802 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15424ed9-3be4-4cc8-99a0-eceba823ac5c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 19:05:17.264860 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-15424ed9-3be4-4cc8-99a0-eceba823ac5c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 19:05:17.264860 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:05:17.264860 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:05:17.264860 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:05:17.264860 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:05:17.264860 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:05:17.264860 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:05:17.264860 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:05:17.264860 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:05:17.264860 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:05:17.264860 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:05:17.264860 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:05:17.264860 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 19:05:17.264860 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 19:05:17.264860 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 19:05:17.264860 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 19:05:17.266575 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 19:05:17.266575 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 19:05:17.266575 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 19:05:17.266575 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 19:05:17.266575 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:05:17.266575 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:05:17.266575 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:05:17.266575 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:05:17.266575 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:05:17.266575 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:05:17.266575 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:05:17.266575 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:05:17.266575 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:05:17.266575 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:05:17.266575 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:05:17.266575 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:05:17.267434 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:05:17.267434 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:05:17.267434 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:05:17.267434 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:05:17.267434 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 19:05:17.267434 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 19:05:17.267434 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 19:05:17.267434 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 19:05:17.267434 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 19:05:17.267434 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 19:05:17.267434 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 19:05:17.267434 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:05:17.269535 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 126/259] 10.222.0.61 () {56 vars in 1096 bytes} [Mon Aug 22 19:05:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 360 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 19:05:17.276020 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15424ed9-3be4-4cc8-99a0-eceba823ac5c None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:17.293931 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5995eda2-59e4-4ccd-be4c-f9d8426d2989 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:17.335669 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7ad678-3380-4f7f-82ef-fafada7fbb15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:17.335669 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7ad678-3380-4f7f-82ef-fafada7fbb15 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:17.335669 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7ad678-3380-4f7f-82ef-fafada7fbb15 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:17.335669 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7ad678-3380-4f7f-82ef-fafada7fbb15 None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:17.336066 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7ad678-3380-4f7f-82ef-fafada7fbb15 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:17.336066 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7ad678-3380-4f7f-82ef-fafada7fbb15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:17.336066 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7ad678-3380-4f7f-82ef-fafada7fbb15 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:17.346490 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 134/260] 10.222.0.61 () {56 vars in 1111 bytes} [Mon Aug 22 19:05:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:17.353174 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7ad678-3380-4f7f-82ef-fafada7fbb15 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:17.419417 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:17.466773 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7ad678-3380-4f7f-82ef-fafada7fbb15 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:17.508241 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89ce1aa-4974-4b45-96e2-c1f447dbf84f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:17.508885 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89ce1aa-4974-4b45-96e2-c1f447dbf84f None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:17.509170 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89ce1aa-4974-4b45-96e2-c1f447dbf84f None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:17.509486 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89ce1aa-4974-4b45-96e2-c1f447dbf84f None admin] PATH_INFO: `/v3/users/61d1f8624beb408cb3bd4cbbde27c9bc` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:17.510635 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89ce1aa-4974-4b45-96e2-c1f447dbf84f None admin] RBAC: Authorizing `identity:get_user(user_id=61d1f8624beb408cb3bd4cbbde27c9bc)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:17.512351 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89ce1aa-4974-4b45-96e2-c1f447dbf84f None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:17.519798 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 135/261] 10.222.0.61 () {56 vars in 1177 bytes} [Mon Aug 22 19:05:17 2022] GET /identity/v3/users/61d1f8624beb408cb3bd4cbbde27c9bc => generated 283 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:17.521196 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89ce1aa-4974-4b45-96e2-c1f447dbf84f None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:17.529644 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15424ed9-3be4-4cc8-99a0-eceba823ac5c None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:17.631986 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89ce1aa-4974-4b45-96e2-c1f447dbf84f None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:17.660367 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dfdcfd-06e1-4624-8d90-f4946b347d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:17.660997 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dfdcfd-06e1-4624-8d90-f4946b347d44 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:17.661277 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dfdcfd-06e1-4624-8d90-f4946b347d44 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:17.661547 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dfdcfd-06e1-4624-8d90-f4946b347d44 None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:17.662224 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dfdcfd-06e1-4624-8d90-f4946b347d44 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:17.662526 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dfdcfd-06e1-4624-8d90-f4946b347d44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:17.662910 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:17.663973 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dfdcfd-06e1-4624-8d90-f4946b347d44 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:17.668175 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb70e95b-c6e5-4435-ac9f-7b8f12ef5608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:17.668918 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb70e95b-c6e5-4435-ac9f-7b8f12ef5608 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:17.669364 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb70e95b-c6e5-4435-ac9f-7b8f12ef5608 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:17.669762 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb70e95b-c6e5-4435-ac9f-7b8f12ef5608 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:17.678096 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 127/262] 10.222.0.61 () {56 vars in 1099 bytes} [Mon Aug 22 19:05:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 402 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:17.683636 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dfdcfd-06e1-4624-8d90-f4946b347d44 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:17.704455 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:17.744360 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb70e95b-c6e5-4435-ac9f-7b8f12ef5608 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:17.746291 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb70e95b-c6e5-4435-ac9f-7b8f12ef5608 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:17.857374 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 136/263] 10.222.0.61 () {58 vars in 1282 bytes} [Mon Aug 22 19:05:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 22 19:05:17.857571 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dfdcfd-06e1-4624-8d90-f4946b347d44 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:17.858911 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb70e95b-c6e5-4435-ac9f-7b8f12ef5608 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:17.892051 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:17.928378 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb401bc4-2f06-4dcf-bf4f-a9272a413deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:17.929039 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb401bc4-2f06-4dcf-bf4f-a9272a413deb None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:17.929318 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb401bc4-2f06-4dcf-bf4f-a9272a413deb None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:17.929592 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb401bc4-2f06-4dcf-bf4f-a9272a413deb None admin] PATH_INFO: `/v3/users/61d1f8624beb408cb3bd4cbbde27c9bc` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:17.930783 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb401bc4-2f06-4dcf-bf4f-a9272a413deb None admin] RBAC: Authorizing `identity:get_user(user_id=61d1f8624beb408cb3bd4cbbde27c9bc)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:17.932256 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb401bc4-2f06-4dcf-bf4f-a9272a413deb None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:17.934171 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 128/264] 10.222.0.61 () {56 vars in 1177 bytes} [Mon Aug 22 19:05:17 2022] GET /identity/v3/users/61d1f8624beb408cb3bd4cbbde27c9bc => generated 283 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:17.940378 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb401bc4-2f06-4dcf-bf4f-a9272a413deb None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:17.966097 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb70e95b-c6e5-4435-ac9f-7b8f12ef5608 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:18.015819 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78710429-8cd9-4924-8cab-23db136194be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:18.016787 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78710429-8cd9-4924-8cab-23db136194be None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:18.017205 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78710429-8cd9-4924-8cab-23db136194be None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:18.017616 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78710429-8cd9-4924-8cab-23db136194be None admin] PATH_INFO: `/v3/projects/47adea686aa0431fb9766ac68f4bf165` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:18.022617 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78710429-8cd9-4924-8cab-23db136194be None admin] RBAC: Authorizing `identity:get_project(project_id=47adea686aa0431fb9766ac68f4bf165)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:18.024513 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78710429-8cd9-4924-8cab-23db136194be None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:18.026355 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 137/265] 10.222.0.61 () {56 vars in 1186 bytes} [Mon Aug 22 19:05:17 2022] GET /identity/v3/projects/47adea686aa0431fb9766ac68f4bf165 => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 19:05:18.031780 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:18.033592 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78710429-8cd9-4924-8cab-23db136194be None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:18.075377 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb401bc4-2f06-4dcf-bf4f-a9272a413deb None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:18.137169 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:18.137794 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e205f8d1-2499-4587-bcfb-61cd605f3417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:18.138422 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e205f8d1-2499-4587-bcfb-61cd605f3417 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:18.138701 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e205f8d1-2499-4587-bcfb-61cd605f3417 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:18.138975 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e205f8d1-2499-4587-bcfb-61cd605f3417 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:18.155890 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78710429-8cd9-4924-8cab-23db136194be None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:18.215191 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b44a4c-b977-466a-b35e-c1d65e0c76ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:18.216074 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b44a4c-b977-466a-b35e-c1d65e0c76ad None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:18.216491 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b44a4c-b977-466a-b35e-c1d65e0c76ad None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:18.217015 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b44a4c-b977-466a-b35e-c1d65e0c76ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:18.218030 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b44a4c-b977-466a-b35e-c1d65e0c76ad None admin] RBAC: Adding query filter params (role.id=9f0952608a704dfca5fdbe4046331f21, scope.project.id=47adea686aa0431fb9766ac68f4bf165, user.id=61d1f8624beb408cb3bd4cbbde27c9bc) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:18.218548 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b44a4c-b977-466a-b35e-c1d65e0c76ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:18.220767 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b44a4c-b977-466a-b35e-c1d65e0c76ad None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:18.227112 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 138/266] 10.222.0.61 () {56 vars in 1374 bytes} [Mon Aug 22 19:05:18 2022] GET /identity/v3/role_assignments?user.id=61d1f8624beb408cb3bd4cbbde27c9bc&scope.project.id=47adea686aa0431fb9766ac68f4bf165&role.id=9f0952608a704dfca5fdbe4046331f21 => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:18.304168 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e205f8d1-2499-4587-bcfb-61cd605f3417 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:18.305870 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e205f8d1-2499-4587-bcfb-61cd605f3417 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:18.395302 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:18.421274 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 129/267] 10.222.0.61 () {58 vars in 1282 bytes} [Mon Aug 22 19:05:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 22 19:05:18.428745 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b44a4c-b977-466a-b35e-c1d65e0c76ad None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:18.544360 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b44a4c-b977-466a-b35e-c1d65e0c76ad None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:18.554690 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02c9565-0a22-4c55-b451-ffc8369c30ed None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:18.556089 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02c9565-0a22-4c55-b451-ffc8369c30ed None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:18.556592 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02c9565-0a22-4c55-b451-ffc8369c30ed None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:18.558708 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 130/268] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:05:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:05:18.567697 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2fb2ad-985d-4315-8e33-f2554afb06e3 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:18.568410 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2fb2ad-985d-4315-8e33-f2554afb06e3 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:18.568885 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2fb2ad-985d-4315-8e33-f2554afb06e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:18.590531 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-ea2fb2ad-985d-4315-8e33-f2554afb06e3 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:05:18.592457 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-ea2fb2ad-985d-4315-8e33-f2554afb06e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:05:18.606180 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc8f3ca-13aa-4694-a7a9-669eb6bc4a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:18.607217 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc8f3ca-13aa-4694-a7a9-669eb6bc4a55 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:18.607717 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc8f3ca-13aa-4694-a7a9-669eb6bc4a55 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:18.608201 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc8f3ca-13aa-4694-a7a9-669eb6bc4a55 None admin] PATH_INFO: `/v3/projects/47adea686aa0431fb9766ac68f4bf165` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:18.610192 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc8f3ca-13aa-4694-a7a9-669eb6bc4a55 None admin] RBAC: Authorizing `identity:get_project(project_id=47adea686aa0431fb9766ac68f4bf165)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:18.611876 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc8f3ca-13aa-4694-a7a9-669eb6bc4a55 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:18.614109 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 139/269] 10.222.0.61 () {56 vars in 1186 bytes} [Mon Aug 22 19:05:18 2022] GET /identity/v3/projects/47adea686aa0431fb9766ac68f4bf165 => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:18.620269 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc8f3ca-13aa-4694-a7a9-669eb6bc4a55 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:18.650774 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:18.744602 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc8f3ca-13aa-4694-a7a9-669eb6bc4a55 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:18.748484 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 131/270] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:05:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 19:05:18.765426 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c505adb9-e27e-434f-8ec7-ff19f3773559 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:18.765783 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c505adb9-e27e-434f-8ec7-ff19f3773559 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:18.766055 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c505adb9-e27e-434f-8ec7-ff19f3773559 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:18.767676 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 132/271] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:05:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:05:18.772459 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0de897e-ca83-4e43-b35e-c468a0cdb7c1 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:18.772736 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0de897e-ca83-4e43-b35e-c468a0cdb7c1 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:18.773008 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0de897e-ca83-4e43-b35e-c468a0cdb7c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:18.785115 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e86695-eb47-4f66-b8b9-4a978adfe9fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:18.786065 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e86695-eb47-4f66-b8b9-4a978adfe9fb None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:18.786554 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e86695-eb47-4f66-b8b9-4a978adfe9fb None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:18.787018 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e86695-eb47-4f66-b8b9-4a978adfe9fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:18.788042 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e86695-eb47-4f66-b8b9-4a978adfe9fb None admin] RBAC: Adding query filter params (role.id=163c30c3b3c246beb849ce176f02d9da, scope.project.id=47adea686aa0431fb9766ac68f4bf165, user.id=61d1f8624beb408cb3bd4cbbde27c9bc) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:18.788616 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e86695-eb47-4f66-b8b9-4a978adfe9fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:18.790361 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e86695-eb47-4f66-b8b9-4a978adfe9fb None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:18.798813 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 140/272] 10.222.0.61 () {56 vars in 1374 bytes} [Mon Aug 22 19:05:18 2022] GET /identity/v3/role_assignments?user.id=61d1f8624beb408cb3bd4cbbde27c9bc&scope.project.id=47adea686aa0431fb9766ac68f4bf165&role.id=163c30c3b3c246beb849ce176f02d9da => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 19:05:18.808856 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-d0de897e-ca83-4e43-b35e-c468a0cdb7c1 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:05:18.810084 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-d0de897e-ca83-4e43-b35e-c468a0cdb7c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:05:18.902161 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 133/273] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:05:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 19:05:19.053484 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e86695-eb47-4f66-b8b9-4a978adfe9fb None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:19.161231 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:19.175516 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e86695-eb47-4f66-b8b9-4a978adfe9fb None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:19.212437 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab73374-5b70-4f9f-b90f-c186ddf4f251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:19.213445 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab73374-5b70-4f9f-b90f-c186ddf4f251 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:19.213931 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab73374-5b70-4f9f-b90f-c186ddf4f251 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:19.214395 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab73374-5b70-4f9f-b90f-c186ddf4f251 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:19.215726 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab73374-5b70-4f9f-b90f-c186ddf4f251 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:19.217543 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab73374-5b70-4f9f-b90f-c186ddf4f251 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:19.219901 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 141/274] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:05:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:19.225509 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0de897e-ca83-4e43-b35e-c468a0cdb7c1 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:19.237620 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab73374-5b70-4f9f-b90f-c186ddf4f251 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:19.265974 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:19.359473 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0de897e-ca83-4e43-b35e-c468a0cdb7c1 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:19.366440 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab73374-5b70-4f9f-b90f-c186ddf4f251 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:19.408393 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81033f61-6734-4445-b77f-6d93d39421fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:19.409927 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81033f61-6734-4445-b77f-6d93d39421fd None admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:19.410439 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81033f61-6734-4445-b77f-6d93d39421fd None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:19.411017 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81033f61-6734-4445-b77f-6d93d39421fd None admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:19.411980 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81033f61-6734-4445-b77f-6d93d39421fd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:19.413838 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81033f61-6734-4445-b77f-6d93d39421fd None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:19.414509 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32de60b-f45a-4732-922c-7e7a1aec74c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:19.415147 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32de60b-f45a-4732-922c-7e7a1aec74c9 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:19.415436 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32de60b-f45a-4732-922c-7e7a1aec74c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:19.416477 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32de60b-f45a-4732-922c-7e7a1aec74c9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:19.428259 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32de60b-f45a-4732-922c-7e7a1aec74c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 19:05:19.435012 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-e32de60b-f45a-4732-922c-7e7a1aec74c9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 22 19:05:19.435012 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:05:19.435012 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:05:19.435012 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:05:19.435012 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:05:19.435012 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:05:19.435012 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:05:19.435012 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:05:19.435012 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:05:19.435012 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:05:19.435012 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:05:19.435012 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:05:19.435012 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 19:05:19.435012 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 19:05:19.435012 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 19:05:19.435012 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 19:05:19.436143 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 19:05:19.436143 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 19:05:19.436143 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 19:05:19.436143 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 19:05:19.436143 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:05:19.436143 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:05:19.436143 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:05:19.436143 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:05:19.436143 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:05:19.436143 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:05:19.436143 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:05:19.436143 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:05:19.436143 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:05:19.436143 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:05:19.436143 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:05:19.436143 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:05:19.437008 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:05:19.437008 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:05:19.437008 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:05:19.437008 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:05:19.437008 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 19:05:19.437008 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 19:05:19.437008 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 19:05:19.437008 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 19:05:19.437008 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 19:05:19.437008 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 19:05:19.437008 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 22 19:05:19.437008 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:05:19.440830 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 142/275] 10.222.0.61 () {56 vars in 1099 bytes} [Mon Aug 22 19:05:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 19:05:19.449166 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 134/276] 10.222.0.61 () {60 vars in 1132 bytes} [Mon Aug 22 19:05:19 2022] POST /identity/v3/users => generated 293 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:05:19.449979 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32de60b-f45a-4732-922c-7e7a1aec74c9 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:19.474572 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:19.571499 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32de60b-f45a-4732-922c-7e7a1aec74c9 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:19.598351 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5e85a6-b8f8-4d1b-9d26-bbd7534b52f4 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:19.598906 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5e85a6-b8f8-4d1b-9d26-bbd7534b52f4 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:19.598906 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5e85a6-b8f8-4d1b-9d26-bbd7534b52f4 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:19.600503 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 135/277] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:05:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:05:19.605669 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ba6680-3d62-444f-b70e-a63c3ff48604 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:19.605951 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ba6680-3d62-444f-b70e-a63c3ff48604 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:19.606222 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ba6680-3d62-444f-b70e-a63c3ff48604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:19.611217 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cc77a0-cf7f-4f13-8d25-c42c946958f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:19.612210 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cc77a0-cf7f-4f13-8d25-c42c946958f5 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:19.612494 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cc77a0-cf7f-4f13-8d25-c42c946958f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:19.612763 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cc77a0-cf7f-4f13-8d25-c42c946958f5 None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:19.613625 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cc77a0-cf7f-4f13-8d25-c42c946958f5 None admin] RBAC: Adding query filter params (name=member) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:19.614187 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cc77a0-cf7f-4f13-8d25-c42c946958f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:19.616454 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cc77a0-cf7f-4f13-8d25-c42c946958f5 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:19.626035 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 143/278] 10.222.0.61 () {56 vars in 1101 bytes} [Mon Aug 22 19:05:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:19.627002 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-53ba6680-3d62-444f-b70e-a63c3ff48604 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:05:19.628266 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-53ba6680-3d62-444f-b70e-a63c3ff48604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:05:19.632475 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cc77a0-cf7f-4f13-8d25-c42c946958f5 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:19.712885 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 136/279] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:05:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 19:05:19.727512 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:19.752927 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cc77a0-cf7f-4f13-8d25-c42c946958f5 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:19.790060 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f01e11-f104-4e54-9cdb-13781f27a59c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:19.790700 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f01e11-f104-4e54-9cdb-13781f27a59c None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:19.790978 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f01e11-f104-4e54-9cdb-13781f27a59c None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:19.791277 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f01e11-f104-4e54-9cdb-13781f27a59c None admin] PATH_INFO: `/v3/users/530b42ec90c34495b83d1e96d6db9536` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:19.797786 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38881df-d90a-4310-90c3-d88e9de3f5b1 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:19.798341 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38881df-d90a-4310-90c3-d88e9de3f5b1 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:19.798841 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38881df-d90a-4310-90c3-d88e9de3f5b1 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:19.803684 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 137/280] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:05:19 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 19:05:19.806944 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f01e11-f104-4e54-9cdb-13781f27a59c None admin] RBAC: Authorizing `identity:get_user(user_id=530b42ec90c34495b83d1e96d6db9536)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:19.808588 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f01e11-f104-4e54-9cdb-13781f27a59c None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:19.809012 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e516cb-8307-4b24-b37b-ac888cfa0685 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:19.809491 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e516cb-8307-4b24-b37b-ac888cfa0685 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:19.810000 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e516cb-8307-4b24-b37b-ac888cfa0685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:19.810376 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 144/281] 10.222.0.61 () {56 vars in 1177 bytes} [Mon Aug 22 19:05:19 2022] GET /identity/v3/users/530b42ec90c34495b83d1e96d6db9536 => generated 293 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 19:05:19.816646 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f01e11-f104-4e54-9cdb-13781f27a59c None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:19.840756 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-76e516cb-8307-4b24-b37b-ac888cfa0685 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:05:19.841965 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-76e516cb-8307-4b24-b37b-ac888cfa0685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:05:19.873410 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:19.938596 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f01e11-f104-4e54-9cdb-13781f27a59c None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:19.946273 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 138/282] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:05:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 19:05:19.984836 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733fb0c2-cbae-4530-8bfd-74674bfbeff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:19.985444 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733fb0c2-cbae-4530-8bfd-74674bfbeff0 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:19.985717 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733fb0c2-cbae-4530-8bfd-74674bfbeff0 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:19.985986 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733fb0c2-cbae-4530-8bfd-74674bfbeff0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:19.986714 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733fb0c2-cbae-4530-8bfd-74674bfbeff0 None admin] RBAC: Adding query filter params (role.id=74a40bc819cb42eea7e966a4b2a016f1, user.id=530b42ec90c34495b83d1e96d6db9536, scope.system=all) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:19.987017 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733fb0c2-cbae-4530-8bfd-74674bfbeff0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:19.988965 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733fb0c2-cbae-4530-8bfd-74674bfbeff0 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:19.991707 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:19.994932 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 145/283] 10.222.0.61 () {56 vars in 1308 bytes} [Mon Aug 22 19:05:19 2022] GET /identity/v3/role_assignments?user.id=530b42ec90c34495b83d1e96d6db9536&scope.system=all&role.id=74a40bc819cb42eea7e966a4b2a016f1 => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 19:05:20.013144 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e516cb-8307-4b24-b37b-ac888cfa0685 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:20.169098 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e516cb-8307-4b24-b37b-ac888cfa0685 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:20.207581 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa27de1f-a99d-44bf-842b-52800627770c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:20.208613 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa27de1f-a99d-44bf-842b-52800627770c None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:20.209097 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa27de1f-a99d-44bf-842b-52800627770c None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:20.209556 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa27de1f-a99d-44bf-842b-52800627770c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:20.221933 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa27de1f-a99d-44bf-842b-52800627770c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 19:05:20.231602 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-aa27de1f-a99d-44bf-842b-52800627770c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 22 19:05:20.231602 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:05:20.231602 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:05:20.231602 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:05:20.231602 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:05:20.231602 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:05:20.231602 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:05:20.231602 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:05:20.231602 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:05:20.231602 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:05:20.231602 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:05:20.231602 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:05:20.231602 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 19:05:20.231602 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 19:05:20.231602 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 19:05:20.231602 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 19:05:20.232539 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 19:05:20.232539 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 19:05:20.232539 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 19:05:20.232539 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 19:05:20.232539 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:05:20.232539 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:05:20.232539 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:05:20.232539 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:05:20.232539 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:05:20.232539 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:05:20.232539 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:05:20.232539 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:05:20.232539 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:05:20.232539 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:05:20.232539 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:05:20.232539 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:05:20.233334 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:05:20.233334 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:05:20.233334 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:05:20.233334 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:05:20.233334 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 19:05:20.233334 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 19:05:20.233334 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 19:05:20.233334 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 19:05:20.233334 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 19:05:20.233334 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 19:05:20.233334 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 22 19:05:20.233334 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:05:20.236205 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 139/284] 10.222.0.61 () {56 vars in 1099 bytes} [Mon Aug 22 19:05:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:05:20.242209 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733fb0c2-cbae-4530-8bfd-74674bfbeff0 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:20.319810 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa27de1f-a99d-44bf-842b-52800627770c None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:20.338871 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:20.385625 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733fb0c2-cbae-4530-8bfd-74674bfbeff0 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:20.451066 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0783c797-fedb-4f09-919b-5081342f6c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:20.452145 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0783c797-fedb-4f09-919b-5081342f6c24 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:20.453976 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0783c797-fedb-4f09-919b-5081342f6c24 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:20.454520 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0783c797-fedb-4f09-919b-5081342f6c24 None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:20.456366 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0783c797-fedb-4f09-919b-5081342f6c24 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:20.460097 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0783c797-fedb-4f09-919b-5081342f6c24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:20.461710 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0783c797-fedb-4f09-919b-5081342f6c24 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:20.479435 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 146/285] 10.222.0.61 () {56 vars in 1101 bytes} [Mon Aug 22 19:05:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:20.484693 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0783c797-fedb-4f09-919b-5081342f6c24 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:20.495204 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa27de1f-a99d-44bf-842b-52800627770c None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:20.540486 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54c7d08-6a58-4788-8731-808f69032376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:20.541419 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54c7d08-6a58-4788-8731-808f69032376 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:20.541867 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54c7d08-6a58-4788-8731-808f69032376 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:20.542309 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54c7d08-6a58-4788-8731-808f69032376 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:20.561309 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54c7d08-6a58-4788-8731-808f69032376 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 19:05:20.569298 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-a54c7d08-6a58-4788-8731-808f69032376 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 22 19:05:20.569298 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:05:20.569298 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:05:20.569298 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:05:20.569298 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:05:20.569298 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:05:20.569298 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:05:20.569298 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:05:20.569298 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:05:20.569298 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:05:20.569298 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:05:20.569298 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:05:20.569298 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 19:05:20.569298 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 19:05:20.569298 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 19:05:20.569298 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 19:05:20.570159 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 19:05:20.570159 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 19:05:20.570159 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 19:05:20.570159 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 19:05:20.570159 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:05:20.570159 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:05:20.570159 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:05:20.570159 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:05:20.570159 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:05:20.570159 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:05:20.570159 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:05:20.570159 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:05:20.570159 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:05:20.570159 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:05:20.570159 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:05:20.570159 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:05:20.570954 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:05:20.570954 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:05:20.570954 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:05:20.570954 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:05:20.570954 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 19:05:20.570954 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 19:05:20.570954 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 19:05:20.570954 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 19:05:20.570954 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 19:05:20.570954 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 19:05:20.570954 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 22 19:05:20.570954 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:05:20.577923 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 140/286] 10.222.0.61 () {56 vars in 1099 bytes} [Mon Aug 22 19:05:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:05:20.585332 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54c7d08-6a58-4788-8731-808f69032376 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:20.613258 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:20.691318 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0783c797-fedb-4f09-919b-5081342f6c24 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:20.732866 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:20.738123 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8044f0e6-95bb-4ddb-8e79-b102ac2c49ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:20.740189 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8044f0e6-95bb-4ddb-8e79-b102ac2c49ab None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:20.740609 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8044f0e6-95bb-4ddb-8e79-b102ac2c49ab None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:20.741042 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8044f0e6-95bb-4ddb-8e79-b102ac2c49ab None admin] PATH_INFO: `/v3/users/e06c4ef6c379460bba9b1190a073b191` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:20.742330 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8044f0e6-95bb-4ddb-8e79-b102ac2c49ab None admin] RBAC: Authorizing `identity:get_user(user_id=e06c4ef6c379460bba9b1190a073b191)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:20.744059 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8044f0e6-95bb-4ddb-8e79-b102ac2c49ab None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:20.746255 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 147/287] 10.222.0.61 () {56 vars in 1177 bytes} [Mon Aug 22 19:05:20 2022] GET /identity/v3/users/e06c4ef6c379460bba9b1190a073b191 => generated 297 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 19:05:20.753018 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8044f0e6-95bb-4ddb-8e79-b102ac2c49ab None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:20.795183 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54c7d08-6a58-4788-8731-808f69032376 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:20.854266 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecf8e04-3fab-4516-be73-6e21c02c6b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:20.855628 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecf8e04-3fab-4516-be73-6e21c02c6b39 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:20.856145 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecf8e04-3fab-4516-be73-6e21c02c6b39 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:20.856630 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecf8e04-3fab-4516-be73-6e21c02c6b39 None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:20.857823 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecf8e04-3fab-4516-be73-6e21c02c6b39 None admin] RBAC: Adding query filter params (name=member) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:20.858833 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecf8e04-3fab-4516-be73-6e21c02c6b39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:20.861353 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecf8e04-3fab-4516-be73-6e21c02c6b39 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:20.877460 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 141/288] 10.222.0.61 () {56 vars in 1101 bytes} [Mon Aug 22 19:05:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 19:05:20.883281 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecf8e04-3fab-4516-be73-6e21c02c6b39 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:20.938698 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8044f0e6-95bb-4ddb-8e79-b102ac2c49ab None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:20.971146 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:20.978326 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98519ebd-977a-42c5-bcd4-9b9cae87a5c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:20.979938 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98519ebd-977a-42c5-bcd4-9b9cae87a5c0 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:20.980484 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98519ebd-977a-42c5-bcd4-9b9cae87a5c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:20.981179 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98519ebd-977a-42c5-bcd4-9b9cae87a5c0 None admin] PATH_INFO: `/v3/projects/47adea686aa0431fb9766ac68f4bf165` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:20.982542 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98519ebd-977a-42c5-bcd4-9b9cae87a5c0 None admin] RBAC: Authorizing `identity:get_project(project_id=47adea686aa0431fb9766ac68f4bf165)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:20.984885 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98519ebd-977a-42c5-bcd4-9b9cae87a5c0 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:20.986894 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 148/289] 10.222.0.61 () {56 vars in 1186 bytes} [Mon Aug 22 19:05:20 2022] GET /identity/v3/projects/47adea686aa0431fb9766ac68f4bf165 => generated 302 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:20.993788 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98519ebd-977a-42c5-bcd4-9b9cae87a5c0 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:21.078813 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecf8e04-3fab-4516-be73-6e21c02c6b39 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:21.146592 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1217747-cf4f-4291-91af-e85a13655643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:21.147101 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1217747-cf4f-4291-91af-e85a13655643 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:21.147393 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1217747-cf4f-4291-91af-e85a13655643 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:21.147676 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1217747-cf4f-4291-91af-e85a13655643 None admin] PATH_INFO: `/v3/users/17b0d9642212479abbf4a557562ba107` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:21.152391 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1217747-cf4f-4291-91af-e85a13655643 None admin] RBAC: Authorizing `identity:get_user(user_id=17b0d9642212479abbf4a557562ba107)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:21.153836 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1217747-cf4f-4291-91af-e85a13655643 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:21.155675 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 142/290] 10.222.0.61 () {56 vars in 1177 bytes} [Mon Aug 22 19:05:20 2022] GET /identity/v3/users/17b0d9642212479abbf4a557562ba107 => generated 297 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:21.161456 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1217747-cf4f-4291-91af-e85a13655643 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:21.165040 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:21.187972 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98519ebd-977a-42c5-bcd4-9b9cae87a5c0 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:21.266408 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461b7381-01bb-4688-9fe5-636064e166e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:21.267002 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461b7381-01bb-4688-9fe5-636064e166e7 None admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:21.267278 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461b7381-01bb-4688-9fe5-636064e166e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:21.267551 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461b7381-01bb-4688-9fe5-636064e166e7 None admin] PATH_INFO: `/v3/projects/47adea686aa0431fb9766ac68f4bf165/users/e06c4ef6c379460bba9b1190a073b191/roles/3f7ffde3fddd46caa6e49c24a6f07673` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:21.274146 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461b7381-01bb-4688-9fe5-636064e166e7 None admin] RBAC: Authorizing `identity:create_grant(project_id=47adea686aa0431fb9766ac68f4bf165, user_id=e06c4ef6c379460bba9b1190a073b191, role_id=3f7ffde3fddd46caa6e49c24a6f07673)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:21.275601 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461b7381-01bb-4688-9fe5-636064e166e7 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:21.295546 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 149/291] 10.222.0.61 () {58 vars in 1439 bytes} [Mon Aug 22 19:05:20 2022] PUT /identity/v3/projects/47adea686aa0431fb9766ac68f4bf165/users/e06c4ef6c379460bba9b1190a073b191/roles/3f7ffde3fddd46caa6e49c24a6f07673 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:05:21.325946 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1217747-cf4f-4291-91af-e85a13655643 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:21.382440 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca9652d-1682-4d53-8924-9fa910c8c1cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:21.383343 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca9652d-1682-4d53-8924-9fa910c8c1cf None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:21.383742 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca9652d-1682-4d53-8924-9fa910c8c1cf None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:21.384132 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca9652d-1682-4d53-8924-9fa910c8c1cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:21.479524 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca9652d-1682-4d53-8924-9fa910c8c1cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:21.481066 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca9652d-1682-4d53-8924-9fa910c8c1cf None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:21.482947 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:21.588490 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 143/292] 10.222.0.61 () {58 vars in 1282 bytes} [Mon Aug 22 19:05:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 22 19:05:21.594232 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461b7381-01bb-4688-9fe5-636064e166e7 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:21.673989 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:21.731567 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461b7381-01bb-4688-9fe5-636064e166e7 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:21.780900 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09425602-1607-4dac-a12a-d6a9bac6417e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:21.782379 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09425602-1607-4dac-a12a-d6a9bac6417e None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:21.782853 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09425602-1607-4dac-a12a-d6a9bac6417e None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:21.783319 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09425602-1607-4dac-a12a-d6a9bac6417e None admin] PATH_INFO: `/v3/projects/47adea686aa0431fb9766ac68f4bf165` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:21.784692 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09425602-1607-4dac-a12a-d6a9bac6417e None admin] RBAC: Authorizing `identity:get_project(project_id=47adea686aa0431fb9766ac68f4bf165)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:21.786416 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09425602-1607-4dac-a12a-d6a9bac6417e None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:21.788338 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 150/293] 10.222.0.61 () {56 vars in 1186 bytes} [Mon Aug 22 19:05:21 2022] GET /identity/v3/projects/47adea686aa0431fb9766ac68f4bf165 => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:21.796746 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca9652d-1682-4d53-8924-9fa910c8c1cf None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:21.935687 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca9652d-1682-4d53-8924-9fa910c8c1cf None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:21.937278 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:21.982341 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9842ae11-03a0-430d-9dcf-a32af8edc509 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:21.982685 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9842ae11-03a0-430d-9dcf-a32af8edc509 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:21.982895 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9842ae11-03a0-430d-9dcf-a32af8edc509 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:21.985367 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 151/294] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:05:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:05:21.989588 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358c6466-3a8e-4785-b3b0-c057ec0a9a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:21.990207 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358c6466-3a8e-4785-b3b0-c057ec0a9a21 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:21.990482 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358c6466-3a8e-4785-b3b0-c057ec0a9a21 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:21.990748 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358c6466-3a8e-4785-b3b0-c057ec0a9a21 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:21.991486 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358c6466-3a8e-4785-b3b0-c057ec0a9a21 None admin] RBAC: Adding query filter params (role.id=74a40bc819cb42eea7e966a4b2a016f1, scope.project.id=47adea686aa0431fb9766ac68f4bf165, user.id=17b0d9642212479abbf4a557562ba107) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:21.991590 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a742a1d-4216-4124-848d-5b0d090081cb None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:21.991819 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358c6466-3a8e-4785-b3b0-c057ec0a9a21 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:21.991923 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a742a1d-4216-4124-848d-5b0d090081cb None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:21.992129 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a742a1d-4216-4124-848d-5b0d090081cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:21.993258 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358c6466-3a8e-4785-b3b0-c057ec0a9a21 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:22.002004 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 144/295] 10.222.0.61 () {56 vars in 1374 bytes} [Mon Aug 22 19:05:21 2022] GET /identity/v3/role_assignments?user.id=17b0d9642212479abbf4a557562ba107&scope.project.id=47adea686aa0431fb9766ac68f4bf165&role.id=74a40bc819cb42eea7e966a4b2a016f1 => generated 608 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:22.028872 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-0a742a1d-4216-4124-848d-5b0d090081cb None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:05:22.030137 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-0a742a1d-4216-4124-848d-5b0d090081cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:05:22.047677 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3834973-230e-4376-b051-170ebeb1322a None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:22.047982 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3834973-230e-4376-b051-170ebeb1322a None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:22.048229 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3834973-230e-4376-b051-170ebeb1322a None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:22.049689 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 145/296] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:05:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 19:05:22.055996 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98efaf0-864f-4fe3-a648-20dbdb0dc1fb None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:22.056276 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98efaf0-864f-4fe3-a648-20dbdb0dc1fb None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:22.056546 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98efaf0-864f-4fe3-a648-20dbdb0dc1fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:22.092776 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-e98efaf0-864f-4fe3-a648-20dbdb0dc1fb None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:05:22.094566 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-e98efaf0-864f-4fe3-a648-20dbdb0dc1fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:05:22.150134 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 152/297] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:05:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 19:05:22.210110 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 146/298] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:05:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 19:05:22.353622 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50104e8-12f7-488b-941e-1ba97e8c4667 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:22.353935 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50104e8-12f7-488b-941e-1ba97e8c4667 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:22.354172 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50104e8-12f7-488b-941e-1ba97e8c4667 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:22.355862 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 153/299] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:05:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:05:22.360832 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab3cb02-2b5f-452f-8e0b-2807212ff55a None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:22.361114 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab3cb02-2b5f-452f-8e0b-2807212ff55a None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:22.361385 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab3cb02-2b5f-452f-8e0b-2807212ff55a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:22.386329 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-2ab3cb02-2b5f-452f-8e0b-2807212ff55a None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:05:22.387516 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-2ab3cb02-2b5f-452f-8e0b-2807212ff55a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:05:22.463411 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 147/300] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:05:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 19:05:22.470538 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50104e8-12f7-488b-941e-1ba97e8c4667 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:22.506767 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab3cb02-2b5f-452f-8e0b-2807212ff55a None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:22.539859 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:22.556571 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:22.612127 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50104e8-12f7-488b-941e-1ba97e8c4667 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:22.624212 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab3cb02-2b5f-452f-8e0b-2807212ff55a None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:22.648056 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b7c302-942a-4ec0-bd3e-763165a4e17e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:22.648688 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b7c302-942a-4ec0-bd3e-763165a4e17e None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:22.648967 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b7c302-942a-4ec0-bd3e-763165a4e17e None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:22.649236 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b7c302-942a-4ec0-bd3e-763165a4e17e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:22.660374 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b7c302-942a-4ec0-bd3e-763165a4e17e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 19:05:22.662035 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642eea73-6d13-456a-a6f0-1dd5f96604b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:22.662935 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642eea73-6d13-456a-a6f0-1dd5f96604b8 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:22.663421 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642eea73-6d13-456a-a6f0-1dd5f96604b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:22.663919 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642eea73-6d13-456a-a6f0-1dd5f96604b8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:22.665211 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642eea73-6d13-456a-a6f0-1dd5f96604b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:22.665576 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-f4b7c302-942a-4ec0-bd3e-763165a4e17e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 22 19:05:22.665576 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:05:22.665576 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:05:22.665576 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:05:22.665576 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:05:22.665576 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:05:22.665576 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:05:22.665576 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:05:22.665576 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:05:22.665576 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:05:22.665576 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:05:22.665576 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:05:22.665576 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 19:05:22.665576 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 19:05:22.665576 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 19:05:22.665576 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 19:05:22.666429 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 19:05:22.666429 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 19:05:22.666429 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 19:05:22.666429 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 19:05:22.666429 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:05:22.666429 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:05:22.666429 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:05:22.666429 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:05:22.666429 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:05:22.666429 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:05:22.666429 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:05:22.666429 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:05:22.666429 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:05:22.666429 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:05:22.666429 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:05:22.666429 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:05:22.667208 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:05:22.667208 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:05:22.667208 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:05:22.667208 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:05:22.667208 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 19:05:22.667208 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 19:05:22.667208 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 19:05:22.667208 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 19:05:22.667208 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 19:05:22.667208 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 19:05:22.667208 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 22 19:05:22.667208 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:05:22.667208 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 154/301] 10.222.0.61 () {56 vars in 1099 bytes} [Mon Aug 22 19:05:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:05:22.669910 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642eea73-6d13-456a-a6f0-1dd5f96604b8 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:22.671530 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 148/302] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:05:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 19:05:22.672474 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b7c302-942a-4ec0-bd3e-763165a4e17e None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:22.676676 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642eea73-6d13-456a-a6f0-1dd5f96604b8 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:22.775527 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:22.784183 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642eea73-6d13-456a-a6f0-1dd5f96604b8 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:22.787622 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b7c302-942a-4ec0-bd3e-763165a4e17e None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:22.809695 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:22.821039 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934b4aa2-c038-4a0f-a013-899c7086b4ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:22.821791 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934b4aa2-c038-4a0f-a013-899c7086b4ff None admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:22.822073 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934b4aa2-c038-4a0f-a013-899c7086b4ff None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:22.822412 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934b4aa2-c038-4a0f-a013-899c7086b4ff None admin] PATH_INFO: `/v3/groups` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:22.823041 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934b4aa2-c038-4a0f-a013-899c7086b4ff None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:22.824495 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934b4aa2-c038-4a0f-a013-899c7086b4ff None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:22.830029 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efdace4-a8a8-4063-9340-f266872b7a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:22.830999 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efdace4-a8a8-4063-9340-f266872b7a0c None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:22.831281 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efdace4-a8a8-4063-9340-f266872b7a0c None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:22.831699 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efdace4-a8a8-4063-9340-f266872b7a0c None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:22.832661 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efdace4-a8a8-4063-9340-f266872b7a0c None admin] RBAC: Adding query filter params (name=reader) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:22.833037 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efdace4-a8a8-4063-9340-f266872b7a0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:22.834656 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efdace4-a8a8-4063-9340-f266872b7a0c None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:22.841355 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 149/303] 10.222.0.61 () {60 vars in 1134 bytes} [Mon Aug 22 19:05:22 2022] POST /identity/v3/groups => generated 229 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 19:05:22.844253 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 155/304] 10.222.0.61 () {56 vars in 1101 bytes} [Mon Aug 22 19:05:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 19:05:22.844965 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efdace4-a8a8-4063-9340-f266872b7a0c None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:22.856752 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934b4aa2-c038-4a0f-a013-899c7086b4ff None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:22.974614 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934b4aa2-c038-4a0f-a013-899c7086b4ff None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:22.979079 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efdace4-a8a8-4063-9340-f266872b7a0c None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:23.018101 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca455890-93b5-4821-85ac-a61f9a1f12da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:23.018418 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a88a90f-2a6a-421d-8da7-c9bbb376b1e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:23.018740 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca455890-93b5-4821-85ac-a61f9a1f12da None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:23.018978 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a88a90f-2a6a-421d-8da7-c9bbb376b1e1 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:23.019084 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca455890-93b5-4821-85ac-a61f9a1f12da None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:23.019252 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a88a90f-2a6a-421d-8da7-c9bbb376b1e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:23.019344 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca455890-93b5-4821-85ac-a61f9a1f12da None admin] PATH_INFO: `/v3/users/df134b61aa624201adb600512bc5c3c0` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:23.019526 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a88a90f-2a6a-421d-8da7-c9bbb376b1e1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:23.031364 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a88a90f-2a6a-421d-8da7-c9bbb376b1e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 19:05:23.036130 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-8a88a90f-2a6a-421d-8da7-c9bbb376b1e1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 22 19:05:23.036130 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:05:23.036130 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:05:23.036130 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:05:23.036130 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:05:23.036130 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:05:23.036130 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:05:23.036130 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:05:23.036130 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:05:23.036130 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:05:23.036130 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:05:23.036130 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:05:23.036130 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 19:05:23.036130 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 19:05:23.036130 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 19:05:23.036130 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 19:05:23.037068 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 19:05:23.037068 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 19:05:23.037068 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 19:05:23.037068 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 19:05:23.037068 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:05:23.037068 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:05:23.037068 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:05:23.037068 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:05:23.037068 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:05:23.037068 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:05:23.037068 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:05:23.037068 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:05:23.037068 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:05:23.037068 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:05:23.037068 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:05:23.037068 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:05:23.037851 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:05:23.037851 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:05:23.037851 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:05:23.037851 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:05:23.037851 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 19:05:23.037851 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 19:05:23.037851 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 19:05:23.037851 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 19:05:23.037851 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 19:05:23.037851 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 19:05:23.037851 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 22 19:05:23.037851 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:05:23.038444 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca455890-93b5-4821-85ac-a61f9a1f12da None admin] RBAC: Authorizing `identity:get_user(user_id=df134b61aa624201adb600512bc5c3c0)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:23.038522 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 156/305] 10.222.0.61 () {56 vars in 1099 bytes} [Mon Aug 22 19:05:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:05:23.038631 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca455890-93b5-4821-85ac-a61f9a1f12da None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:23.042455 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 150/306] 10.222.0.61 () {56 vars in 1177 bytes} [Mon Aug 22 19:05:22 2022] GET /identity/v3/users/df134b61aa624201adb600512bc5c3c0 => generated 293 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:23.045833 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a88a90f-2a6a-421d-8da7-c9bbb376b1e1 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:23.047477 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca455890-93b5-4821-85ac-a61f9a1f12da None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:23.066858 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:23.124048 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:23.161343 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a88a90f-2a6a-421d-8da7-c9bbb376b1e1 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:23.176158 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca455890-93b5-4821-85ac-a61f9a1f12da None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:23.206619 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08edb33a-0774-4f20-9a7f-d8696371a60a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:23.207349 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08edb33a-0774-4f20-9a7f-d8696371a60a None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:23.207633 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08edb33a-0774-4f20-9a7f-d8696371a60a None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:23.207951 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08edb33a-0774-4f20-9a7f-d8696371a60a None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:23.208791 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08edb33a-0774-4f20-9a7f-d8696371a60a None admin] RBAC: Adding query filter params (name=member) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:23.209098 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08edb33a-0774-4f20-9a7f-d8696371a60a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:23.211115 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08edb33a-0774-4f20-9a7f-d8696371a60a None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:23.214595 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc224ede-bbba-42e8-9004-a690d8d2cc98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:23.215369 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc224ede-bbba-42e8-9004-a690d8d2cc98 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:23.215815 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc224ede-bbba-42e8-9004-a690d8d2cc98 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:23.216275 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc224ede-bbba-42e8-9004-a690d8d2cc98 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:23.217527 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc224ede-bbba-42e8-9004-a690d8d2cc98 None admin] RBAC: Adding query filter params (role.id=3f7ffde3fddd46caa6e49c24a6f07673, user.id=df134b61aa624201adb600512bc5c3c0, scope.system=all) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:23.217974 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc224ede-bbba-42e8-9004-a690d8d2cc98 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:23.219576 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc224ede-bbba-42e8-9004-a690d8d2cc98 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:23.220251 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 157/307] 10.222.0.61 () {56 vars in 1101 bytes} [Mon Aug 22 19:05:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 19:05:23.225527 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08edb33a-0774-4f20-9a7f-d8696371a60a None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:23.228067 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 151/308] 10.222.0.61 () {56 vars in 1308 bytes} [Mon Aug 22 19:05:23 2022] GET /identity/v3/role_assignments?user.id=df134b61aa624201adb600512bc5c3c0&scope.system=all&role.id=3f7ffde3fddd46caa6e49c24a6f07673 => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 19:05:23.339071 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:23.343103 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08edb33a-0774-4f20-9a7f-d8696371a60a None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:23.387179 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb24dbaf-eabd-4cd3-849c-9ee1b0a63b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:23.387846 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb24dbaf-eabd-4cd3-849c-9ee1b0a63b37 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:23.388122 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb24dbaf-eabd-4cd3-849c-9ee1b0a63b37 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:23.388399 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb24dbaf-eabd-4cd3-849c-9ee1b0a63b37 None admin] PATH_INFO: `/v3/users/530b42ec90c34495b83d1e96d6db9536` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:23.390397 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb24dbaf-eabd-4cd3-849c-9ee1b0a63b37 None admin] RBAC: Authorizing `identity:get_user(user_id=530b42ec90c34495b83d1e96d6db9536)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:23.391111 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb24dbaf-eabd-4cd3-849c-9ee1b0a63b37 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:23.392909 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 158/309] 10.222.0.61 () {56 vars in 1177 bytes} [Mon Aug 22 19:05:23 2022] GET /identity/v3/users/530b42ec90c34495b83d1e96d6db9536 => generated 293 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:23.398960 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc224ede-bbba-42e8-9004-a690d8d2cc98 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:23.486694 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:23.511498 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc224ede-bbba-42e8-9004-a690d8d2cc98 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:23.551471 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4a1e06-63f4-4892-b6a8-611307a011c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:23.552161 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4a1e06-63f4-4892-b6a8-611307a011c6 None admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:23.552441 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4a1e06-63f4-4892-b6a8-611307a011c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:23.552712 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4a1e06-63f4-4892-b6a8-611307a011c6 None admin] PATH_INFO: `/v3/system/users/530b42ec90c34495b83d1e96d6db9536/roles/74a40bc819cb42eea7e966a4b2a016f1` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:23.554166 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4a1e06-63f4-4892-b6a8-611307a011c6 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=530b42ec90c34495b83d1e96d6db9536, role_id=74a40bc819cb42eea7e966a4b2a016f1)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:23.555539 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4a1e06-63f4-4892-b6a8-611307a011c6 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:23.568408 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 152/310] 10.222.0.61 () {58 vars in 1334 bytes} [Mon Aug 22 19:05:23 2022] PUT /identity/v3/system/users/530b42ec90c34495b83d1e96d6db9536/roles/74a40bc819cb42eea7e966a4b2a016f1 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:05:23.573312 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e749119-a68b-42fa-a9f6-d15279395038 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:23.573592 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e749119-a68b-42fa-a9f6-d15279395038 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:23.573867 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e749119-a68b-42fa-a9f6-d15279395038 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:23.575589 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 159/311] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:05:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:05:23.581991 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eaaaa3-1bf0-40b1-bee5-16d1939d8038 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:23.582279 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eaaaa3-1bf0-40b1-bee5-16d1939d8038 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:23.582554 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eaaaa3-1bf0-40b1-bee5-16d1939d8038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:23.602991 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-f0eaaaa3-1bf0-40b1-bee5-16d1939d8038 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:05:23.604183 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-f0eaaaa3-1bf0-40b1-bee5-16d1939d8038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:05:23.681587 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 153/312] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:05:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 19:05:23.946491 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e749119-a68b-42fa-a9f6-d15279395038 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:24.053341 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e749119-a68b-42fa-a9f6-d15279395038 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:24.087288 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797b112e-4ce2-4d4e-9109-45da77326d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:24.088264 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797b112e-4ce2-4d4e-9109-45da77326d12 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:24.088684 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797b112e-4ce2-4d4e-9109-45da77326d12 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:24.089093 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797b112e-4ce2-4d4e-9109-45da77326d12 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:24.100069 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797b112e-4ce2-4d4e-9109-45da77326d12 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 19:05:24.105648 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-797b112e-4ce2-4d4e-9109-45da77326d12 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 22 19:05:24.105648 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:05:24.105648 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:05:24.105648 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:05:24.105648 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:05:24.105648 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:05:24.105648 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:05:24.105648 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:05:24.105648 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:05:24.105648 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:05:24.105648 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:05:24.105648 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:05:24.105648 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 19:05:24.105648 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 19:05:24.105648 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 19:05:24.105648 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 19:05:24.106692 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 19:05:24.106692 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 19:05:24.106692 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 19:05:24.106692 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 19:05:24.106692 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:05:24.106692 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:05:24.106692 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:05:24.106692 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:05:24.106692 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:05:24.106692 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:05:24.106692 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:05:24.106692 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:05:24.106692 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:05:24.106692 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:05:24.106692 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:05:24.106692 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:05:24.107715 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:05:24.107715 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:05:24.107715 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:05:24.107715 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:05:24.107715 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 19:05:24.107715 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 19:05:24.107715 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 19:05:24.107715 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 19:05:24.107715 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 19:05:24.107715 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 19:05:24.107715 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 22 19:05:24.107715 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:05:24.110022 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 160/313] 10.222.0.61 () {56 vars in 1099 bytes} [Mon Aug 22 19:05:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:05:24.120772 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eaaaa3-1bf0-40b1-bee5-16d1939d8038 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:24.191814 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:24.225196 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eaaaa3-1bf0-40b1-bee5-16d1939d8038 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:24.260435 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57553c0b-3aa5-4699-a447-7a2c1db50183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:24.261048 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57553c0b-3aa5-4699-a447-7a2c1db50183 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:24.261323 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57553c0b-3aa5-4699-a447-7a2c1db50183 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:24.261594 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57553c0b-3aa5-4699-a447-7a2c1db50183 None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:24.262270 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57553c0b-3aa5-4699-a447-7a2c1db50183 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:24.262570 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57553c0b-3aa5-4699-a447-7a2c1db50183 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:24.263979 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57553c0b-3aa5-4699-a447-7a2c1db50183 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:24.273253 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 154/314] 10.222.0.61 () {56 vars in 1101 bytes} [Mon Aug 22 19:05:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:24.278574 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797b112e-4ce2-4d4e-9109-45da77326d12 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:24.308248 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:24.384102 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797b112e-4ce2-4d4e-9109-45da77326d12 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:24.421409 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81abd04-b735-4a39-8780-37e2ebbe11a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:24.422128 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81abd04-b735-4a39-8780-37e2ebbe11a7 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:24.422410 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81abd04-b735-4a39-8780-37e2ebbe11a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:24.422683 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81abd04-b735-4a39-8780-37e2ebbe11a7 None admin] PATH_INFO: `/v3/users/e06c4ef6c379460bba9b1190a073b191` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:24.423908 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81abd04-b735-4a39-8780-37e2ebbe11a7 None admin] RBAC: Authorizing `identity:get_user(user_id=e06c4ef6c379460bba9b1190a073b191)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:24.425350 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81abd04-b735-4a39-8780-37e2ebbe11a7 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:24.427257 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 161/315] 10.222.0.61 () {56 vars in 1177 bytes} [Mon Aug 22 19:05:24 2022] GET /identity/v3/users/e06c4ef6c379460bba9b1190a073b191 => generated 297 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:24.432837 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57553c0b-3aa5-4699-a447-7a2c1db50183 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:24.538325 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57553c0b-3aa5-4699-a447-7a2c1db50183 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:24.572044 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:24.573323 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdc2d75-659b-45f2-b3be-fa44e2214fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:24.573938 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdc2d75-659b-45f2-b3be-fa44e2214fba None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:24.574225 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdc2d75-659b-45f2-b3be-fa44e2214fba None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:24.574507 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdc2d75-659b-45f2-b3be-fa44e2214fba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:24.651262 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdc2d75-659b-45f2-b3be-fa44e2214fba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:24.652822 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdc2d75-659b-45f2-b3be-fa44e2214fba None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:24.758138 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 155/316] 10.222.0.61 () {58 vars in 1282 bytes} [Mon Aug 22 19:05:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 19:05:24.764113 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81abd04-b735-4a39-8780-37e2ebbe11a7 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:24.868615 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81abd04-b735-4a39-8780-37e2ebbe11a7 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:24.872231 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:24.876977 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66f2fd6-f57d-42ce-843c-0276720289ff None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:24.877257 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66f2fd6-f57d-42ce-843c-0276720289ff None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:24.877525 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66f2fd6-f57d-42ce-843c-0276720289ff None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:24.880539 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 156/317] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:05:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:05:24.884620 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1d52cd-55f4-445d-b1cd-4ee95507b0ef None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:24.884899 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1d52cd-55f4-445d-b1cd-4ee95507b0ef None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:24.885171 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1d52cd-55f4-445d-b1cd-4ee95507b0ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:24.905162 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-8b1d52cd-55f4-445d-b1cd-4ee95507b0ef None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:05:24.906775 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-8b1d52cd-55f4-445d-b1cd-4ee95507b0ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:05:24.907256 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f846dbe0-4020-4950-81ec-0642799be9d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:24.907935 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f846dbe0-4020-4950-81ec-0642799be9d6 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:24.908214 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f846dbe0-4020-4950-81ec-0642799be9d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:24.908484 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f846dbe0-4020-4950-81ec-0642799be9d6 None admin] PATH_INFO: `/v3/projects/47adea686aa0431fb9766ac68f4bf165` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:24.910771 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f846dbe0-4020-4950-81ec-0642799be9d6 None admin] RBAC: Authorizing `identity:get_project(project_id=47adea686aa0431fb9766ac68f4bf165)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:24.912714 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f846dbe0-4020-4950-81ec-0642799be9d6 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:24.916376 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 162/318] 10.222.0.61 () {56 vars in 1186 bytes} [Mon Aug 22 19:05:24 2022] GET /identity/v3/projects/47adea686aa0431fb9766ac68f4bf165 => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:24.922494 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f846dbe0-4020-4950-81ec-0642799be9d6 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:24.939373 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:24.998303 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 157/319] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:05:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 19:05:25.034548 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f846dbe0-4020-4950-81ec-0642799be9d6 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:25.070241 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6dcc8e-1b79-4ba2-bdea-4cb843c04170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:25.070858 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6dcc8e-1b79-4ba2-bdea-4cb843c04170 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:25.071137 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6dcc8e-1b79-4ba2-bdea-4cb843c04170 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:25.071410 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6dcc8e-1b79-4ba2-bdea-4cb843c04170 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:25.072216 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6dcc8e-1b79-4ba2-bdea-4cb843c04170 None admin] RBAC: Adding query filter params (role.id=3f7ffde3fddd46caa6e49c24a6f07673, scope.project.id=47adea686aa0431fb9766ac68f4bf165, user.id=e06c4ef6c379460bba9b1190a073b191) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:25.072527 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6dcc8e-1b79-4ba2-bdea-4cb843c04170 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:25.073965 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6dcc8e-1b79-4ba2-bdea-4cb843c04170 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:25.080393 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 163/320] 10.222.0.61 () {56 vars in 1374 bytes} [Mon Aug 22 19:05:24 2022] GET /identity/v3/role_assignments?user.id=e06c4ef6c379460bba9b1190a073b191&scope.project.id=47adea686aa0431fb9766ac68f4bf165&role.id=3f7ffde3fddd46caa6e49c24a6f07673 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 19:05:25.268925 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5715591-df4e-4c69-93f8-d2ced450db62 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:25.269916 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5715591-df4e-4c69-93f8-d2ced450db62 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:25.270415 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5715591-df4e-4c69-93f8-d2ced450db62 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:25.272026 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 158/321] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:05:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:05:25.273302 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6dcc8e-1b79-4ba2-bdea-4cb843c04170 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:25.277350 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03dd70b-e699-49cf-a2be-bce7a9c7e9f5 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:25.277634 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03dd70b-e699-49cf-a2be-bce7a9c7e9f5 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:25.277913 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03dd70b-e699-49cf-a2be-bce7a9c7e9f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:25.297619 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-b03dd70b-e699-49cf-a2be-bce7a9c7e9f5 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:05:25.298809 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-b03dd70b-e699-49cf-a2be-bce7a9c7e9f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:05:25.313565 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:25.374703 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 159/322] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:05:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 19:05:25.390339 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6dcc8e-1b79-4ba2-bdea-4cb843c04170 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:25.425765 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5fa055-8832-4915-965b-4ef5357ba98a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:25.426701 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5fa055-8832-4915-965b-4ef5357ba98a None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:25.427199 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5fa055-8832-4915-965b-4ef5357ba98a None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:25.427711 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5fa055-8832-4915-965b-4ef5357ba98a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:25.429014 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5fa055-8832-4915-965b-4ef5357ba98a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:25.430645 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5fa055-8832-4915-965b-4ef5357ba98a None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:25.432446 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 164/323] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:05:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:25.438098 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03dd70b-e699-49cf-a2be-bce7a9c7e9f5 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:25.534201 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:25.540341 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2ef874-f4c1-46f8-9a8f-cb44570392a9 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:25.540621 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2ef874-f4c1-46f8-9a8f-cb44570392a9 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:25.540887 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2ef874-f4c1-46f8-9a8f-cb44570392a9 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:25.542218 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 165/324] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:05:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 19:05:25.545047 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03dd70b-e699-49cf-a2be-bce7a9c7e9f5 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:25.547115 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36f189e-5d61-4d1c-ba67-988432a36252 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:25.547402 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36f189e-5d61-4d1c-ba67-988432a36252 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:25.547707 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36f189e-5d61-4d1c-ba67-988432a36252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:25.566181 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-b36f189e-5d61-4d1c-ba67-988432a36252 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:05:25.567371 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-b36f189e-5d61-4d1c-ba67-988432a36252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:05:25.581837 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9988de-de0a-4827-9671-d2d8c08c45ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:25.582741 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9988de-de0a-4827-9671-d2d8c08c45ae None admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:25.583205 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9988de-de0a-4827-9671-d2d8c08c45ae None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:25.583673 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9988de-de0a-4827-9671-d2d8c08c45ae None admin] PATH_INFO: `/v3/groups` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:25.584616 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9988de-de0a-4827-9671-d2d8c08c45ae None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:25.586338 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9988de-de0a-4827-9671-d2d8c08c45ae None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:25.602722 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 160/325] 10.222.0.61 () {60 vars in 1134 bytes} [Mon Aug 22 19:05:25 2022] POST /identity/v3/groups => generated 226 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 19:05:25.651480 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9988de-de0a-4827-9671-d2d8c08c45ae None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:25.653741 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 166/326] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:05:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 19:05:25.766355 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9988de-de0a-4827-9671-d2d8c08c45ae None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:25.802491 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4fcf3b-02ee-4511-a0cb-fc284218b737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:25.803196 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4fcf3b-02ee-4511-a0cb-fc284218b737 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:25.803477 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4fcf3b-02ee-4511-a0cb-fc284218b737 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:25.803799 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4fcf3b-02ee-4511-a0cb-fc284218b737 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:25.814671 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4fcf3b-02ee-4511-a0cb-fc284218b737 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 19:05:25.819947 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-7b4fcf3b-02ee-4511-a0cb-fc284218b737 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 22 19:05:25.819947 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:05:25.819947 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:05:25.819947 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:05:25.819947 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:05:25.819947 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:05:25.819947 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:05:25.819947 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:05:25.819947 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:05:25.819947 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:05:25.819947 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:05:25.819947 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:05:25.819947 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 19:05:25.819947 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 19:05:25.819947 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 19:05:25.819947 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 19:05:25.820821 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 19:05:25.820821 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 19:05:25.820821 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 19:05:25.820821 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 19:05:25.820821 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:05:25.820821 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:05:25.820821 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:05:25.820821 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:05:25.820821 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:05:25.820821 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:05:25.820821 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:05:25.820821 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:05:25.820821 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:05:25.820821 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:05:25.820821 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:05:25.820821 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:05:25.821626 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:05:25.821626 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:05:25.821626 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:05:25.821626 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:05:25.821626 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 19:05:25.821626 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 19:05:25.821626 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 19:05:25.821626 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 19:05:25.821626 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 19:05:25.821626 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 19:05:25.821626 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 22 19:05:25.821626 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:05:25.821626 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 161/327] 10.222.0.61 () {56 vars in 1099 bytes} [Mon Aug 22 19:05:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:05:25.828799 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36f189e-5d61-4d1c-ba67-988432a36252 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:25.860694 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:25.923733 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4fcf3b-02ee-4511-a0cb-fc284218b737 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:25.942452 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:25.944109 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36f189e-5d61-4d1c-ba67-988432a36252 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:25.987689 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751768b0-5b19-4bd0-84a7-2c0c7f4f8d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:25.988295 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751768b0-5b19-4bd0-84a7-2c0c7f4f8d66 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:25.988574 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751768b0-5b19-4bd0-84a7-2c0c7f4f8d66 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:25.989108 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751768b0-5b19-4bd0-84a7-2c0c7f4f8d66 None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:25.990030 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751768b0-5b19-4bd0-84a7-2c0c7f4f8d66 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:25.990145 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751768b0-5b19-4bd0-84a7-2c0c7f4f8d66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:25.991445 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751768b0-5b19-4bd0-84a7-2c0c7f4f8d66 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:26.003090 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 167/328] 10.222.0.61 () {56 vars in 1101 bytes} [Mon Aug 22 19:05:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:26.010500 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751768b0-5b19-4bd0-84a7-2c0c7f4f8d66 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:26.049215 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4fcf3b-02ee-4511-a0cb-fc284218b737 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:26.088678 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc27919-9d8a-4a5f-880a-2e3fc9e4b5c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:26.089617 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc27919-9d8a-4a5f-880a-2e3fc9e4b5c7 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:26.090070 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc27919-9d8a-4a5f-880a-2e3fc9e4b5c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:26.090486 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc27919-9d8a-4a5f-880a-2e3fc9e4b5c7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:26.102705 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc27919-9d8a-4a5f-880a-2e3fc9e4b5c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 19:05:26.104523 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:26.113153 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-efc27919-9d8a-4a5f-880a-2e3fc9e4b5c7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 22 19:05:26.113153 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:05:26.113153 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:05:26.113153 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:05:26.113153 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:05:26.113153 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:05:26.113153 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:05:26.113153 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:05:26.113153 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:05:26.113153 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:05:26.113153 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:05:26.113153 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:05:26.113153 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 19:05:26.113153 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 19:05:26.113153 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 19:05:26.113153 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 19:05:26.114712 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 19:05:26.114712 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 19:05:26.114712 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 19:05:26.114712 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 19:05:26.114712 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:05:26.114712 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:05:26.114712 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:05:26.114712 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:05:26.114712 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:05:26.114712 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:05:26.114712 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:05:26.114712 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:05:26.114712 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:05:26.114712 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:05:26.114712 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:05:26.114712 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:05:26.117519 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:05:26.117519 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:05:26.117519 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:05:26.117519 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:05:26.117519 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 19:05:26.117519 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 19:05:26.117519 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 19:05:26.117519 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 19:05:26.117519 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 19:05:26.117519 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 19:05:26.117519 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 22 19:05:26.117519 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:05:26.117519 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 162/329] 10.222.0.61 () {56 vars in 1099 bytes} [Mon Aug 22 19:05:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 19:05:26.119967 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751768b0-5b19-4bd0-84a7-2c0c7f4f8d66 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:26.120865 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc27919-9d8a-4a5f-880a-2e3fc9e4b5c7 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:26.189787 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231c0a23-e0d8-4574-b78f-e58548e02aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:26.190509 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231c0a23-e0d8-4574-b78f-e58548e02aac None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:26.190738 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231c0a23-e0d8-4574-b78f-e58548e02aac None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:26.191012 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231c0a23-e0d8-4574-b78f-e58548e02aac None admin] PATH_INFO: `/v3/users/df134b61aa624201adb600512bc5c3c0` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:26.192348 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231c0a23-e0d8-4574-b78f-e58548e02aac None admin] RBAC: Authorizing `identity:get_user(user_id=df134b61aa624201adb600512bc5c3c0)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:26.193719 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231c0a23-e0d8-4574-b78f-e58548e02aac None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:26.196977 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 168/330] 10.222.0.61 () {56 vars in 1177 bytes} [Mon Aug 22 19:05:26 2022] GET /identity/v3/users/df134b61aa624201adb600512bc5c3c0 => generated 293 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:26.202318 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231c0a23-e0d8-4574-b78f-e58548e02aac None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:26.216294 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:26.254593 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc27919-9d8a-4a5f-880a-2e3fc9e4b5c7 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:26.293538 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c30690-ee04-4df5-accd-983c70e828f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:26.295129 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c30690-ee04-4df5-accd-983c70e828f5 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:26.295657 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c30690-ee04-4df5-accd-983c70e828f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:26.296590 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c30690-ee04-4df5-accd-983c70e828f5 None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:26.298006 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c30690-ee04-4df5-accd-983c70e828f5 None admin] RBAC: Adding query filter params (name=member) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:26.298584 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c30690-ee04-4df5-accd-983c70e828f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:26.300634 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c30690-ee04-4df5-accd-983c70e828f5 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:26.310814 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 163/331] 10.222.0.61 () {56 vars in 1101 bytes} [Mon Aug 22 19:05:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:26.316572 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c30690-ee04-4df5-accd-983c70e828f5 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:26.336616 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231c0a23-e0d8-4574-b78f-e58548e02aac None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:26.384421 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46db7ff-c55c-4749-bf2d-ebcabf0f81f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:26.384992 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46db7ff-c55c-4749-bf2d-ebcabf0f81f6 None admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:26.385275 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46db7ff-c55c-4749-bf2d-ebcabf0f81f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:26.385550 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46db7ff-c55c-4749-bf2d-ebcabf0f81f6 None admin] PATH_INFO: `/v3/system/users/df134b61aa624201adb600512bc5c3c0/roles/3f7ffde3fddd46caa6e49c24a6f07673` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:26.387708 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46db7ff-c55c-4749-bf2d-ebcabf0f81f6 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=df134b61aa624201adb600512bc5c3c0, role_id=3f7ffde3fddd46caa6e49c24a6f07673)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:26.389099 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46db7ff-c55c-4749-bf2d-ebcabf0f81f6 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:26.405971 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 169/332] 10.222.0.61 () {58 vars in 1334 bytes} [Mon Aug 22 19:05:26 2022] PUT /identity/v3/system/users/df134b61aa624201adb600512bc5c3c0/roles/3f7ffde3fddd46caa6e49c24a6f07673 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:05:26.451130 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c30690-ee04-4df5-accd-983c70e828f5 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:26.493009 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfd25e0-2a3b-4512-b2a8-23841213069e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:26.494236 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfd25e0-2a3b-4512-b2a8-23841213069e None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:26.494811 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfd25e0-2a3b-4512-b2a8-23841213069e None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:26.495280 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfd25e0-2a3b-4512-b2a8-23841213069e None admin] PATH_INFO: `/v3/users/530b42ec90c34495b83d1e96d6db9536` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:26.497024 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfd25e0-2a3b-4512-b2a8-23841213069e None admin] RBAC: Authorizing `identity:get_user(user_id=530b42ec90c34495b83d1e96d6db9536)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:26.498720 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfd25e0-2a3b-4512-b2a8-23841213069e None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:26.501285 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 164/333] 10.222.0.61 () {56 vars in 1177 bytes} [Mon Aug 22 19:05:26 2022] GET /identity/v3/users/530b42ec90c34495b83d1e96d6db9536 => generated 293 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 19:05:26.506977 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46db7ff-c55c-4749-bf2d-ebcabf0f81f6 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:26.536742 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:26.629634 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46db7ff-c55c-4749-bf2d-ebcabf0f81f6 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:26.669361 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300a5c86-85f2-4d8e-a52e-ad99b8e5e2f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:26.669977 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300a5c86-85f2-4d8e-a52e-ad99b8e5e2f4 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:26.670258 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300a5c86-85f2-4d8e-a52e-ad99b8e5e2f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:26.670530 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300a5c86-85f2-4d8e-a52e-ad99b8e5e2f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:26.671266 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300a5c86-85f2-4d8e-a52e-ad99b8e5e2f4 None admin] RBAC: Adding query filter params (role.id=74a40bc819cb42eea7e966a4b2a016f1, user.id=530b42ec90c34495b83d1e96d6db9536, scope.system=all) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:26.671575 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300a5c86-85f2-4d8e-a52e-ad99b8e5e2f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:26.673669 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300a5c86-85f2-4d8e-a52e-ad99b8e5e2f4 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:26.686976 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 170/334] 10.222.0.61 () {56 vars in 1308 bytes} [Mon Aug 22 19:05:26 2022] GET /identity/v3/role_assignments?user.id=530b42ec90c34495b83d1e96d6db9536&scope.system=all&role.id=74a40bc819cb42eea7e966a4b2a016f1 => generated 510 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 19:05:28.068015 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7db945e-fd6e-46ad-a4c5-7b97110f2422 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:28.068541 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7db945e-fd6e-46ad-a4c5-7b97110f2422 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:28.068541 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7db945e-fd6e-46ad-a4c5-7b97110f2422 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:28.070154 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 165/335] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:05:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:05:28.075945 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db83ba6-c81f-4ff4-ae41-9b77d237d8ed None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:28.076370 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db83ba6-c81f-4ff4-ae41-9b77d237d8ed None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:28.076541 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db83ba6-c81f-4ff4-ae41-9b77d237d8ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:28.098311 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-4db83ba6-c81f-4ff4-ae41-9b77d237d8ed None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:05:28.099576 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-4db83ba6-c81f-4ff4-ae41-9b77d237d8ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:05:28.181382 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 171/336] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:05:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 19:05:28.472223 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7db945e-fd6e-46ad-a4c5-7b97110f2422 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:28.475196 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:28.563380 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aa64c2-a84c-4d6e-85ee-8050742915f3 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:28.563740 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aa64c2-a84c-4d6e-85ee-8050742915f3 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:28.563962 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aa64c2-a84c-4d6e-85ee-8050742915f3 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:28.567108 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 172/337] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:05:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:05:28.575276 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd38937-6029-4c14-a6dd-9aad9a767f15 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:28.575856 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd38937-6029-4c14-a6dd-9aad9a767f15 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:28.576329 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd38937-6029-4c14-a6dd-9aad9a767f15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:28.610250 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7db945e-fd6e-46ad-a4c5-7b97110f2422 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:28.625862 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-1fd38937-6029-4c14-a6dd-9aad9a767f15 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:05:28.627078 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-1fd38937-6029-4c14-a6dd-9aad9a767f15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:05:28.630984 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:28.697960 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459dd5ca-22c3-47ea-95de-658b993ca0e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:28.698608 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459dd5ca-22c3-47ea-95de-658b993ca0e6 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:28.698885 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459dd5ca-22c3-47ea-95de-658b993ca0e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:28.699154 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459dd5ca-22c3-47ea-95de-658b993ca0e6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:28.733647 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459dd5ca-22c3-47ea-95de-658b993ca0e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 19:05:28.739516 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-459dd5ca-22c3-47ea-95de-658b993ca0e6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 22 19:05:28.739516 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:05:28.739516 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:05:28.739516 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:05:28.739516 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:05:28.739516 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:05:28.739516 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:05:28.739516 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:05:28.739516 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:05:28.739516 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:05:28.739516 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:05:28.739516 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:05:28.739516 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 19:05:28.739516 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 19:05:28.739516 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 19:05:28.739516 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 19:05:28.741177 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 19:05:28.741177 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 19:05:28.741177 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 19:05:28.741177 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 19:05:28.741177 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:05:28.741177 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:05:28.741177 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:05:28.741177 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:05:28.741177 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:05:28.741177 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:05:28.741177 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:05:28.741177 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:05:28.741177 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:05:28.741177 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:05:28.741177 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:05:28.741177 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:05:28.742748 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:05:28.742748 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:05:28.742748 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:05:28.742748 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:05:28.742748 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 19:05:28.742748 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 19:05:28.742748 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 19:05:28.742748 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 19:05:28.742748 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 19:05:28.742748 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 19:05:28.742748 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 22 19:05:28.742748 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:05:28.744967 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 166/338] 10.222.0.61 () {56 vars in 1099 bytes} [Mon Aug 22 19:05:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:05:28.753935 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459dd5ca-22c3-47ea-95de-658b993ca0e6 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:28.798580 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 173/339] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:05:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 19:05:28.860009 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:28.887351 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20c4a5f-58ce-484b-9e68-6379507479f9 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:28.887708 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20c4a5f-58ce-484b-9e68-6379507479f9 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:28.888023 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20c4a5f-58ce-484b-9e68-6379507479f9 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:28.891300 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 174/340] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:05:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 19:05:28.895432 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b9b39a-125a-4247-87eb-1d37eb3ffb89 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:28.895739 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b9b39a-125a-4247-87eb-1d37eb3ffb89 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:28.896019 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b9b39a-125a-4247-87eb-1d37eb3ffb89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:28.903451 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459dd5ca-22c3-47ea-95de-658b993ca0e6 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:28.918124 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-77b9b39a-125a-4247-87eb-1d37eb3ffb89 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:05:28.919334 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-77b9b39a-125a-4247-87eb-1d37eb3ffb89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:05:28.960009 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f134e91-3586-42b3-a593-5777b180dfa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:28.960626 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f134e91-3586-42b3-a593-5777b180dfa0 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:28.960905 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f134e91-3586-42b3-a593-5777b180dfa0 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:28.961175 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f134e91-3586-42b3-a593-5777b180dfa0 None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:28.961848 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f134e91-3586-42b3-a593-5777b180dfa0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:28.962151 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f134e91-3586-42b3-a593-5777b180dfa0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:28.963529 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f134e91-3586-42b3-a593-5777b180dfa0 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:28.982381 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 167/341] 10.222.0.61 () {56 vars in 1101 bytes} [Mon Aug 22 19:05:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:28.985578 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f134e91-3586-42b3-a593-5777b180dfa0 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:29.038174 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 175/342] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:05:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 19:05:29.111683 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c9ee21-aaf5-4a6b-b8ab-34b1d4b50579 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:29.111683 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c9ee21-aaf5-4a6b-b8ab-34b1d4b50579 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:29.111683 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c9ee21-aaf5-4a6b-b8ab-34b1d4b50579 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:29.124626 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 176/343] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:05:29 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:05:29.127267 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f134e91-3586-42b3-a593-5777b180dfa0 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:29.133145 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a2d9ad-ff15-48e1-8429-8c9373de4dee None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:29.133695 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a2d9ad-ff15-48e1-8429-8c9373de4dee None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:29.134187 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a2d9ad-ff15-48e1-8429-8c9373de4dee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:29.168024 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:29.183549 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-92a2d9ad-ff15-48e1-8429-8c9373de4dee None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:05:29.184861 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-92a2d9ad-ff15-48e1-8429-8c9373de4dee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:05:29.195303 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc108d6a-27f0-4b7a-b86b-a0b7984f6e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:29.195952 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc108d6a-27f0-4b7a-b86b-a0b7984f6e5c None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:29.196232 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc108d6a-27f0-4b7a-b86b-a0b7984f6e5c None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:29.196504 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc108d6a-27f0-4b7a-b86b-a0b7984f6e5c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:29.304999 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:29.311028 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 177/344] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:05:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 19:05:29.327512 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc108d6a-27f0-4b7a-b86b-a0b7984f6e5c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:29.329227 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc108d6a-27f0-4b7a-b86b-a0b7984f6e5c None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:29.385863 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a2d9ad-ff15-48e1-8429-8c9373de4dee None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:29.498689 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 168/345] 10.222.0.61 () {58 vars in 1282 bytes} [Mon Aug 22 19:05:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 516 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 19:05:29.504782 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc108d6a-27f0-4b7a-b86b-a0b7984f6e5c None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:29.544007 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:29.605916 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a2d9ad-ff15-48e1-8429-8c9373de4dee None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:29.644564 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc108d6a-27f0-4b7a-b86b-a0b7984f6e5c None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:29.696707 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4903e031-72dd-4f59-a4e4-4acf9dcbfb75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:29.697367 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4903e031-72dd-4f59-a4e4-4acf9dcbfb75 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:29.697648 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4903e031-72dd-4f59-a4e4-4acf9dcbfb75 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:29.697924 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4903e031-72dd-4f59-a4e4-4acf9dcbfb75 None admin] PATH_INFO: `/v3/projects/a4fe339e17324d15a22e51626818eecd` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:29.704335 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4903e031-72dd-4f59-a4e4-4acf9dcbfb75 None admin] RBAC: Authorizing `identity:get_project(project_id=a4fe339e17324d15a22e51626818eecd)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:29.705801 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4903e031-72dd-4f59-a4e4-4acf9dcbfb75 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:29.707747 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 169/346] 10.222.0.61 () {56 vars in 1186 bytes} [Mon Aug 22 19:05:29 2022] GET /identity/v3/projects/a4fe339e17324d15a22e51626818eecd => generated 298 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 19:05:29.713465 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4903e031-72dd-4f59-a4e4-4acf9dcbfb75 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:29.728360 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71937c13-6bd0-4d55-b4eb-61ca52c40c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:29.729016 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71937c13-6bd0-4d55-b4eb-61ca52c40c14 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:29.729300 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71937c13-6bd0-4d55-b4eb-61ca52c40c14 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:29.729575 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71937c13-6bd0-4d55-b4eb-61ca52c40c14 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:29.752100 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71937c13-6bd0-4d55-b4eb-61ca52c40c14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 19:05:29.756997 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-71937c13-6bd0-4d55-b4eb-61ca52c40c14 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 22 19:05:29.756997 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:05:29.756997 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:05:29.756997 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:05:29.756997 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:05:29.756997 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:05:29.756997 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:05:29.756997 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:05:29.756997 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:05:29.756997 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:05:29.756997 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:05:29.756997 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:05:29.756997 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 19:05:29.756997 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 19:05:29.756997 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 19:05:29.756997 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 19:05:29.757869 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 19:05:29.757869 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 19:05:29.757869 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 19:05:29.757869 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 19:05:29.757869 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:05:29.757869 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:05:29.757869 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:05:29.757869 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:05:29.757869 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:05:29.757869 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:05:29.757869 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:05:29.757869 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:05:29.757869 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:05:29.757869 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:05:29.757869 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:05:29.757869 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:05:29.758668 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:05:29.758668 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:05:29.758668 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:05:29.758668 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:05:29.758668 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 19:05:29.758668 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 19:05:29.758668 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 19:05:29.758668 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 19:05:29.758668 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 19:05:29.758668 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 19:05:29.758668 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 22 19:05:29.758668 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:05:29.764508 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 178/347] 10.222.0.61 () {56 vars in 1099 bytes} [Mon Aug 22 19:05:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 374 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:05:29.768965 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71937c13-6bd0-4d55-b4eb-61ca52c40c14 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:29.845459 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:29.847727 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:29.857778 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4903e031-72dd-4f59-a4e4-4acf9dcbfb75 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:29.905534 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5418c75b-e2bb-484d-82f7-6046c5bc866e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:29.906243 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5418c75b-e2bb-484d-82f7-6046c5bc866e None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:29.907022 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5418c75b-e2bb-484d-82f7-6046c5bc866e None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:29.907408 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5418c75b-e2bb-484d-82f7-6046c5bc866e None admin] PATH_INFO: `/v3/groups/f21c86cb093e4771896943da49969864` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:29.915528 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71937c13-6bd0-4d55-b4eb-61ca52c40c14 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:29.920158 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5418c75b-e2bb-484d-82f7-6046c5bc866e None admin] RBAC: Authorizing `identity:get_group(group_id=f21c86cb093e4771896943da49969864)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:29.921646 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5418c75b-e2bb-484d-82f7-6046c5bc866e None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:29.923752 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 170/348] 10.222.0.61 () {56 vars in 1180 bytes} [Mon Aug 22 19:05:29 2022] GET /identity/v3/groups/f21c86cb093e4771896943da49969864 => generated 226 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 19:05:29.924545 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5418c75b-e2bb-484d-82f7-6046c5bc866e None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:29.964082 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1826a96-c84c-4787-8392-8e3d8d7baa38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:29.964851 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1826a96-c84c-4787-8392-8e3d8d7baa38 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:29.965128 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1826a96-c84c-4787-8392-8e3d8d7baa38 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:29.965401 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1826a96-c84c-4787-8392-8e3d8d7baa38 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:29.978266 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1826a96-c84c-4787-8392-8e3d8d7baa38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 19:05:29.983531 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-a1826a96-c84c-4787-8392-8e3d8d7baa38 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 22 19:05:29.983531 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:05:29.983531 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:05:29.983531 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:05:29.983531 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:05:29.983531 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:05:29.983531 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:05:29.983531 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:05:29.983531 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:05:29.983531 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:05:29.983531 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:05:29.983531 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:05:29.983531 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 19:05:29.983531 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 19:05:29.983531 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 19:05:29.983531 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 19:05:29.984752 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 19:05:29.984752 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 19:05:29.984752 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 19:05:29.984752 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 19:05:29.984752 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:05:29.984752 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:05:29.984752 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:05:29.984752 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:05:29.984752 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:05:29.984752 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:05:29.984752 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:05:29.984752 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:05:29.984752 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:05:29.984752 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:05:29.984752 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:05:29.984752 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:05:29.985776 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:05:29.985776 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:05:29.985776 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:05:29.985776 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:05:29.985776 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 19:05:29.985776 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 19:05:29.985776 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 19:05:29.985776 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 19:05:29.985776 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 19:05:29.985776 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 19:05:29.985776 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 22 19:05:29.985776 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:05:29.985776 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 179/349] 10.222.0.61 () {56 vars in 1099 bytes} [Mon Aug 22 19:05:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:05:29.985776 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1826a96-c84c-4787-8392-8e3d8d7baa38 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:30.076770 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5418c75b-e2bb-484d-82f7-6046c5bc866e None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:30.123308 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1826a96-c84c-4787-8392-8e3d8d7baa38 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:30.124013 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98628221-cdb9-4c04-86c0-b9f23ad2055b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:30.124673 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98628221-cdb9-4c04-86c0-b9f23ad2055b None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:30.125151 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98628221-cdb9-4c04-86c0-b9f23ad2055b None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:30.125428 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98628221-cdb9-4c04-86c0-b9f23ad2055b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:30.139718 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98628221-cdb9-4c04-86c0-b9f23ad2055b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 19:05:30.146139 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-98628221-cdb9-4c04-86c0-b9f23ad2055b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 19:05:30.146139 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:05:30.146139 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:05:30.146139 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:05:30.146139 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:05:30.146139 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:05:30.146139 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:05:30.146139 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:05:30.146139 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:05:30.146139 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:05:30.146139 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:05:30.146139 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:05:30.146139 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 19:05:30.146139 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 19:05:30.146139 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 19:05:30.146139 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 19:05:30.147029 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 19:05:30.147029 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 19:05:30.147029 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 19:05:30.147029 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 19:05:30.147029 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:05:30.147029 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:05:30.147029 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:05:30.147029 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:05:30.147029 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:05:30.147029 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:05:30.147029 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:05:30.147029 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:05:30.147029 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:05:30.147029 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:05:30.147029 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:05:30.147029 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:05:30.147924 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:05:30.147924 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:05:30.147924 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:05:30.147924 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:05:30.147924 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 19:05:30.147924 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 19:05:30.147924 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 19:05:30.147924 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 19:05:30.147924 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 19:05:30.147924 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 19:05:30.147924 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 19:05:30.147924 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:05:30.147924 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 171/350] 10.222.0.61 () {56 vars in 1114 bytes} [Mon Aug 22 19:05:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:05:30.149618 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98628221-cdb9-4c04-86c0-b9f23ad2055b None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:30.150967 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:30.175528 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67068dbb-704d-48f1-ae3a-27a227f9d6fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:30.176213 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67068dbb-704d-48f1-ae3a-27a227f9d6fc None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:30.176495 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67068dbb-704d-48f1-ae3a-27a227f9d6fc None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:30.176770 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67068dbb-704d-48f1-ae3a-27a227f9d6fc None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:30.177532 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:30.178734 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67068dbb-704d-48f1-ae3a-27a227f9d6fc None admin] RBAC: Adding query filter params (name=member) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:30.179029 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67068dbb-704d-48f1-ae3a-27a227f9d6fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:30.181024 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67068dbb-704d-48f1-ae3a-27a227f9d6fc None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:30.190360 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 180/351] 10.222.0.61 () {56 vars in 1101 bytes} [Mon Aug 22 19:05:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:30.191268 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67068dbb-704d-48f1-ae3a-27a227f9d6fc None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:30.287109 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98628221-cdb9-4c04-86c0-b9f23ad2055b None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:30.313492 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67068dbb-704d-48f1-ae3a-27a227f9d6fc None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:30.330707 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f8c935-8473-417a-9958-0b5afbda2258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:30.331343 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f8c935-8473-417a-9958-0b5afbda2258 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:30.331618 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f8c935-8473-417a-9958-0b5afbda2258 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:30.332153 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f8c935-8473-417a-9958-0b5afbda2258 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:30.332695 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f8c935-8473-417a-9958-0b5afbda2258 None admin] RBAC: Adding query filter params (group.id=f21c86cb093e4771896943da49969864, role.id=74a40bc819cb42eea7e966a4b2a016f1, scope.project.id=a4fe339e17324d15a22e51626818eecd) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:30.333033 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f8c935-8473-417a-9958-0b5afbda2258 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:30.334471 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f8c935-8473-417a-9958-0b5afbda2258 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:30.341239 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 172/352] 10.222.0.61 () {56 vars in 1376 bytes} [Mon Aug 22 19:05:30 2022] GET /identity/v3/role_assignments?group.id=f21c86cb093e4771896943da49969864&scope.project.id=a4fe339e17324d15a22e51626818eecd&role.id=74a40bc819cb42eea7e966a4b2a016f1 => generated 260 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:30.342052 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30c0a58-e152-489b-ab15-35d81819e3fe None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:30.344798 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30c0a58-e152-489b-ab15-35d81819e3fe None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:30.344798 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30c0a58-e152-489b-ab15-35d81819e3fe None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:30.345670 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 173/353] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:05:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:05:30.346220 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30c0a58-e152-489b-ab15-35d81819e3fe None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:30.369825 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4e29a7-7f14-4bce-9dc3-10f988f026c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:30.370892 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4e29a7-7f14-4bce-9dc3-10f988f026c4 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:30.371510 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4e29a7-7f14-4bce-9dc3-10f988f026c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:30.371893 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4e29a7-7f14-4bce-9dc3-10f988f026c4 None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:30.372655 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4e29a7-7f14-4bce-9dc3-10f988f026c4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:30.373031 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4e29a7-7f14-4bce-9dc3-10f988f026c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:30.374555 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4e29a7-7f14-4bce-9dc3-10f988f026c4 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:30.385059 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 181/354] 10.222.0.61 () {56 vars in 1101 bytes} [Mon Aug 22 19:05:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:30.386254 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4e29a7-7f14-4bce-9dc3-10f988f026c4 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:30.452682 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:30.457062 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:30.488141 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30c0a58-e152-489b-ab15-35d81819e3fe None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:30.534405 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c0ee6b-4a76-4910-86cd-a55d4a6291de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:30.535022 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c0ee6b-4a76-4910-86cd-a55d4a6291de None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:30.535296 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c0ee6b-4a76-4910-86cd-a55d4a6291de None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:30.535569 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c0ee6b-4a76-4910-86cd-a55d4a6291de None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:30.536270 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c0ee6b-4a76-4910-86cd-a55d4a6291de None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:30.536569 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c0ee6b-4a76-4910-86cd-a55d4a6291de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:30.537934 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c0ee6b-4a76-4910-86cd-a55d4a6291de None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:30.540746 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4e29a7-7f14-4bce-9dc3-10f988f026c4 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:30.548472 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 174/355] 10.222.0.61 () {56 vars in 1111 bytes} [Mon Aug 22 19:05:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:30.549518 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b21bca4-b8ad-4838-9fd5-442ad70d6ba7 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:30.549799 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b21bca4-b8ad-4838-9fd5-442ad70d6ba7 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:30.550070 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b21bca4-b8ad-4838-9fd5-442ad70d6ba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:30.572668 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-2b21bca4-b8ad-4838-9fd5-442ad70d6ba7 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:05:30.573972 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-2b21bca4-b8ad-4838-9fd5-442ad70d6ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:05:30.584741 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55fd459-4932-4b41-a4ad-6e6fe4927cfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:30.585757 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55fd459-4932-4b41-a4ad-6e6fe4927cfc None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:30.586043 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55fd459-4932-4b41-a4ad-6e6fe4927cfc None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:30.586643 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55fd459-4932-4b41-a4ad-6e6fe4927cfc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:30.669961 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 175/356] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:05:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 19:05:30.669961 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b21bca4-b8ad-4838-9fd5-442ad70d6ba7 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:30.680479 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55fd459-4932-4b41-a4ad-6e6fe4927cfc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:30.682317 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55fd459-4932-4b41-a4ad-6e6fe4927cfc None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:30.733704 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:30.760188 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:30.803985 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 182/357] 10.222.0.61 () {58 vars in 1282 bytes} [Mon Aug 22 19:05:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 19:05:30.807669 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239d4222-5217-4cdf-b853-2a7c1ac4869b None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:30.807669 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239d4222-5217-4cdf-b853-2a7c1ac4869b None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:30.807669 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239d4222-5217-4cdf-b853-2a7c1ac4869b None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:30.807669 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 183/358] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:05:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:05:30.807994 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239d4222-5217-4cdf-b853-2a7c1ac4869b None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:30.815541 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b21bca4-b8ad-4838-9fd5-442ad70d6ba7 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:30.860791 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1dc12a-516a-41b0-8c66-3bb2e7e01dfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:30.861776 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1dc12a-516a-41b0-8c66-3bb2e7e01dfd None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:30.862254 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1dc12a-516a-41b0-8c66-3bb2e7e01dfd None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:30.862724 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1dc12a-516a-41b0-8c66-3bb2e7e01dfd None admin] PATH_INFO: `/v3/users/df134b61aa624201adb600512bc5c3c0` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:30.864502 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1dc12a-516a-41b0-8c66-3bb2e7e01dfd None admin] RBAC: Authorizing `identity:get_user(user_id=df134b61aa624201adb600512bc5c3c0)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:30.866156 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1dc12a-516a-41b0-8c66-3bb2e7e01dfd None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:30.868010 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 176/359] 10.222.0.61 () {56 vars in 1177 bytes} [Mon Aug 22 19:05:30 2022] GET /identity/v3/users/df134b61aa624201adb600512bc5c3c0 => generated 293 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:30.868984 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1dc12a-516a-41b0-8c66-3bb2e7e01dfd None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:30.924112 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239d4222-5217-4cdf-b853-2a7c1ac4869b None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:30.962224 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda83a5b-2772-466b-9fad-050e13ddab8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:30.962828 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda83a5b-2772-466b-9fad-050e13ddab8b None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:30.963104 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda83a5b-2772-466b-9fad-050e13ddab8b None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:30.963371 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda83a5b-2772-466b-9fad-050e13ddab8b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:30.977411 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:30.983617 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1dc12a-516a-41b0-8c66-3bb2e7e01dfd None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:31.022144 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfe8cc0-309a-4679-a740-0ac91fc1992d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:31.023121 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfe8cc0-309a-4679-a740-0ac91fc1992d None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:31.023600 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfe8cc0-309a-4679-a740-0ac91fc1992d None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:31.024194 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfe8cc0-309a-4679-a740-0ac91fc1992d None admin] PATH_INFO: `/v3/projects/47adea686aa0431fb9766ac68f4bf165` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:31.025544 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfe8cc0-309a-4679-a740-0ac91fc1992d None admin] RBAC: Authorizing `identity:get_project(project_id=47adea686aa0431fb9766ac68f4bf165)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:31.027290 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfe8cc0-309a-4679-a740-0ac91fc1992d None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:31.029259 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 177/360] 10.222.0.61 () {56 vars in 1186 bytes} [Mon Aug 22 19:05:30 2022] GET /identity/v3/projects/47adea686aa0431fb9766ac68f4bf165 => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:31.030589 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e37cd7d-8ae9-4f91-b458-c23b45b7d5af None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:31.031080 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e37cd7d-8ae9-4f91-b458-c23b45b7d5af None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:31.031607 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e37cd7d-8ae9-4f91-b458-c23b45b7d5af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:31.041347 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda83a5b-2772-466b-9fad-050e13ddab8b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:31.042950 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda83a5b-2772-466b-9fad-050e13ddab8b None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:31.050647 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-5e37cd7d-8ae9-4f91-b458-c23b45b7d5af None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:05:31.052141 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-5e37cd7d-8ae9-4f91-b458-c23b45b7d5af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:05:31.054450 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:31.137805 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 178/361] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:05:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 19:05:31.138580 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e37cd7d-8ae9-4f91-b458-c23b45b7d5af None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:31.175773 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 184/362] 10.222.0.61 () {58 vars in 1282 bytes} [Mon Aug 22 19:05:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 19:05:31.177242 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda83a5b-2772-466b-9fad-050e13ddab8b None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:31.228812 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:31.287157 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e37cd7d-8ae9-4f91-b458-c23b45b7d5af None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:31.305204 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda83a5b-2772-466b-9fad-050e13ddab8b None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:31.337652 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c107e78d-eb50-4bf0-b5b6-96eaa5258692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:31.338263 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c107e78d-eb50-4bf0-b5b6-96eaa5258692 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:31.338536 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c107e78d-eb50-4bf0-b5b6-96eaa5258692 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:31.338803 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c107e78d-eb50-4bf0-b5b6-96eaa5258692 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:31.339527 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c107e78d-eb50-4bf0-b5b6-96eaa5258692 None admin] RBAC: Adding query filter params (role.id=3f7ffde3fddd46caa6e49c24a6f07673, user.id=df134b61aa624201adb600512bc5c3c0, scope.system=all) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:31.340032 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c107e78d-eb50-4bf0-b5b6-96eaa5258692 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:31.341452 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c107e78d-eb50-4bf0-b5b6-96eaa5258692 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:31.349439 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:31.354835 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceec2a64-25f9-4ef8-a52b-fd08e0a57b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:31.355762 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceec2a64-25f9-4ef8-a52b-fd08e0a57b62 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:31.356040 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceec2a64-25f9-4ef8-a52b-fd08e0a57b62 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:31.356313 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceec2a64-25f9-4ef8-a52b-fd08e0a57b62 None admin] PATH_INFO: `/v3/groups/f21c86cb093e4771896943da49969864` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:31.358382 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceec2a64-25f9-4ef8-a52b-fd08e0a57b62 None admin] RBAC: Authorizing `identity:get_group(group_id=f21c86cb093e4771896943da49969864)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:31.359898 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceec2a64-25f9-4ef8-a52b-fd08e0a57b62 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:31.360239 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 179/363] 10.222.0.61 () {56 vars in 1308 bytes} [Mon Aug 22 19:05:31 2022] GET /identity/v3/role_assignments?user.id=df134b61aa624201adb600512bc5c3c0&scope.system=all&role.id=3f7ffde3fddd46caa6e49c24a6f07673 => generated 510 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:31.361420 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c107e78d-eb50-4bf0-b5b6-96eaa5258692 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:31.361531 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 185/364] 10.222.0.61 () {56 vars in 1180 bytes} [Mon Aug 22 19:05:31 2022] GET /identity/v3/groups/f21c86cb093e4771896943da49969864 => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:31.362283 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceec2a64-25f9-4ef8-a52b-fd08e0a57b62 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:31.477901 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceec2a64-25f9-4ef8-a52b-fd08e0a57b62 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:31.490583 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:31.495072 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c107e78d-eb50-4bf0-b5b6-96eaa5258692 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:31.518093 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314b6d58-5ac1-4b98-bf18-99c796b77a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:31.518794 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314b6d58-5ac1-4b98-bf18-99c796b77a54 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:31.518997 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314b6d58-5ac1-4b98-bf18-99c796b77a54 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:31.519273 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314b6d58-5ac1-4b98-bf18-99c796b77a54 None admin] PATH_INFO: `/v3/projects/a4fe339e17324d15a22e51626818eecd` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:31.520693 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314b6d58-5ac1-4b98-bf18-99c796b77a54 None admin] RBAC: Authorizing `identity:get_project(project_id=a4fe339e17324d15a22e51626818eecd)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:31.521972 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314b6d58-5ac1-4b98-bf18-99c796b77a54 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:31.523799 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 186/365] 10.222.0.61 () {56 vars in 1186 bytes} [Mon Aug 22 19:05:31 2022] GET /identity/v3/projects/a4fe339e17324d15a22e51626818eecd => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:31.524941 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314b6d58-5ac1-4b98-bf18-99c796b77a54 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:31.543756 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad377b3e-4746-4dc4-9a39-6b263a1c6df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:31.544827 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad377b3e-4746-4dc4-9a39-6b263a1c6df7 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:31.545334 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad377b3e-4746-4dc4-9a39-6b263a1c6df7 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:31.545911 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad377b3e-4746-4dc4-9a39-6b263a1c6df7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:31.559090 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad377b3e-4746-4dc4-9a39-6b263a1c6df7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 19:05:31.565422 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-ad377b3e-4746-4dc4-9a39-6b263a1c6df7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 19:05:31.565422 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:05:31.565422 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:05:31.565422 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:05:31.565422 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:05:31.565422 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:05:31.565422 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:05:31.565422 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:05:31.565422 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:05:31.565422 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:05:31.565422 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:05:31.565422 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:05:31.565422 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 19:05:31.565422 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 19:05:31.565422 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 19:05:31.565422 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 19:05:31.566409 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 19:05:31.566409 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 19:05:31.566409 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 19:05:31.566409 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 19:05:31.566409 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:05:31.566409 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:05:31.566409 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:05:31.566409 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:05:31.566409 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:05:31.566409 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:05:31.566409 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:05:31.566409 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:05:31.566409 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:05:31.566409 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:05:31.566409 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:05:31.566409 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:05:31.567219 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:05:31.567219 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:05:31.567219 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:05:31.567219 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:05:31.567219 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 19:05:31.567219 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 19:05:31.567219 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 19:05:31.567219 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 19:05:31.567219 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 19:05:31.567219 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 19:05:31.567219 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 19:05:31.567219 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:05:31.569677 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 180/366] 10.222.0.61 () {56 vars in 1114 bytes} [Mon Aug 22 19:05:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:05:31.575943 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad377b3e-4746-4dc4-9a39-6b263a1c6df7 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:31.635442 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314b6d58-5ac1-4b98-bf18-99c796b77a54 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:31.643011 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:31.676701 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c1f55d-3c75-4c7c-8486-bb9a8a5dc471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:31.677317 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c1f55d-3c75-4c7c-8486-bb9a8a5dc471 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:31.677650 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c1f55d-3c75-4c7c-8486-bb9a8a5dc471 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:31.677927 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c1f55d-3c75-4c7c-8486-bb9a8a5dc471 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:31.678649 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c1f55d-3c75-4c7c-8486-bb9a8a5dc471 None admin] RBAC: Adding query filter params (group.id=f21c86cb093e4771896943da49969864, role.id=74a40bc819cb42eea7e966a4b2a016f1, scope.project.id=47adea686aa0431fb9766ac68f4bf165) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:31.678953 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c1f55d-3c75-4c7c-8486-bb9a8a5dc471 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:31.680868 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c1f55d-3c75-4c7c-8486-bb9a8a5dc471 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:31.688135 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 187/367] 10.222.0.61 () {56 vars in 1376 bytes} [Mon Aug 22 19:05:31 2022] GET /identity/v3/role_assignments?group.id=f21c86cb093e4771896943da49969864&scope.project.id=47adea686aa0431fb9766ac68f4bf165&role.id=74a40bc819cb42eea7e966a4b2a016f1 => generated 260 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:31.689022 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c1f55d-3c75-4c7c-8486-bb9a8a5dc471 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:31.698409 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad377b3e-4746-4dc4-9a39-6b263a1c6df7 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:31.743458 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898dae52-b2d2-4faa-b26f-a25be9639b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:31.744233 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898dae52-b2d2-4faa-b26f-a25be9639b15 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:31.744512 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898dae52-b2d2-4faa-b26f-a25be9639b15 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:31.745648 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898dae52-b2d2-4faa-b26f-a25be9639b15 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:31.758719 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:31.760765 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898dae52-b2d2-4faa-b26f-a25be9639b15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 19:05:31.769857 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-898dae52-b2d2-4faa-b26f-a25be9639b15 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 19:05:31.769857 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:05:31.769857 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:05:31.769857 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:05:31.769857 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:05:31.769857 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:05:31.769857 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:05:31.769857 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:05:31.769857 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:05:31.769857 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:05:31.769857 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:05:31.769857 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:05:31.769857 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 19:05:31.769857 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 19:05:31.769857 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 19:05:31.769857 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 19:05:31.770833 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 19:05:31.770833 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 19:05:31.770833 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 19:05:31.770833 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 19:05:31.770833 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:05:31.770833 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:05:31.770833 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:05:31.770833 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:05:31.770833 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:05:31.770833 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:05:31.770833 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:05:31.770833 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:05:31.770833 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:05:31.770833 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:05:31.770833 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:05:31.770833 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:05:31.771681 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:05:31.771681 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:05:31.771681 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:05:31.771681 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:05:31.771681 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 19:05:31.771681 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 19:05:31.771681 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 19:05:31.771681 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 19:05:31.771681 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 19:05:31.771681 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 19:05:31.771681 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 19:05:31.771681 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:05:31.772306 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 181/368] 10.222.0.61 () {56 vars in 1096 bytes} [Mon Aug 22 19:05:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:05:31.772909 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898dae52-b2d2-4faa-b26f-a25be9639b15 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:31.843517 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c1f55d-3c75-4c7c-8486-bb9a8a5dc471 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:31.885204 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd33acd2-c3f6-47fb-a934-9d939421a78e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:31.887004 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd33acd2-c3f6-47fb-a934-9d939421a78e None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:31.887004 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd33acd2-c3f6-47fb-a934-9d939421a78e None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:31.887004 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd33acd2-c3f6-47fb-a934-9d939421a78e None admin] PATH_INFO: `/v3/groups/f21c86cb093e4771896943da49969864` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:31.887848 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd33acd2-c3f6-47fb-a934-9d939421a78e None admin] RBAC: Authorizing `identity:get_group(group_id=f21c86cb093e4771896943da49969864)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:31.889466 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd33acd2-c3f6-47fb-a934-9d939421a78e None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:31.891956 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 188/369] 10.222.0.61 () {56 vars in 1180 bytes} [Mon Aug 22 19:05:31 2022] GET /identity/v3/groups/f21c86cb093e4771896943da49969864 => generated 226 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:31.892802 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd33acd2-c3f6-47fb-a934-9d939421a78e None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:31.901451 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898dae52-b2d2-4faa-b26f-a25be9639b15 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:31.921383 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:31.942510 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf02190-9d6b-4d83-a61b-86b5ef57fbc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:31.943137 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf02190-9d6b-4d83-a61b-86b5ef57fbc6 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:31.943421 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf02190-9d6b-4d83-a61b-86b5ef57fbc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:31.943713 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf02190-9d6b-4d83-a61b-86b5ef57fbc6 None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:31.944399 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf02190-9d6b-4d83-a61b-86b5ef57fbc6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:31.944706 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf02190-9d6b-4d83-a61b-86b5ef57fbc6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:31.946185 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf02190-9d6b-4d83-a61b-86b5ef57fbc6 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:31.957597 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 182/370] 10.222.0.61 () {56 vars in 1111 bytes} [Mon Aug 22 19:05:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:31.960459 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf02190-9d6b-4d83-a61b-86b5ef57fbc6 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:32.029324 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd33acd2-c3f6-47fb-a934-9d939421a78e None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:32.043137 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:32.074552 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73a2407-e37f-4a09-8884-08c2129596cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:32.075466 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73a2407-e37f-4a09-8884-08c2129596cf None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:32.075909 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf02190-9d6b-4d83-a61b-86b5ef57fbc6 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:32.076152 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73a2407-e37f-4a09-8884-08c2129596cf None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:32.076594 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73a2407-e37f-4a09-8884-08c2129596cf None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:32.077604 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73a2407-e37f-4a09-8884-08c2129596cf None admin] RBAC: Adding query filter params (name=admin) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:32.078643 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73a2407-e37f-4a09-8884-08c2129596cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:32.082188 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73a2407-e37f-4a09-8884-08c2129596cf None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:32.094388 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 189/371] 10.222.0.61 () {56 vars in 1099 bytes} [Mon Aug 22 19:05:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:32.095785 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73a2407-e37f-4a09-8884-08c2129596cf None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:32.127587 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4c3a5e-640e-495c-9fc9-7316233f5215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:32.128274 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4c3a5e-640e-495c-9fc9-7316233f5215 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:32.128532 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4c3a5e-640e-495c-9fc9-7316233f5215 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:32.128866 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4c3a5e-640e-495c-9fc9-7316233f5215 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:32.129665 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4c3a5e-640e-495c-9fc9-7316233f5215 None admin] RBAC: Adding query filter params (group.id=f21c86cb093e4771896943da49969864, role.id=9f0952608a704dfca5fdbe4046331f21, scope.project.id=a4fe339e17324d15a22e51626818eecd) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:32.129976 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4c3a5e-640e-495c-9fc9-7316233f5215 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:32.131440 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4c3a5e-640e-495c-9fc9-7316233f5215 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:32.137911 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 183/372] 10.222.0.61 () {56 vars in 1376 bytes} [Mon Aug 22 19:05:31 2022] GET /identity/v3/role_assignments?group.id=f21c86cb093e4771896943da49969864&scope.project.id=a4fe339e17324d15a22e51626818eecd&role.id=9f0952608a704dfca5fdbe4046331f21 => generated 260 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:32.138796 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4c3a5e-640e-495c-9fc9-7316233f5215 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:32.220020 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:32.260757 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73a2407-e37f-4a09-8884-08c2129596cf None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:32.307357 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a3361b-ce35-46e6-ad82-222f2b209652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:32.307960 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a3361b-ce35-46e6-ad82-222f2b209652 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:32.308239 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a3361b-ce35-46e6-ad82-222f2b209652 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:32.308510 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a3361b-ce35-46e6-ad82-222f2b209652 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:32.329777 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4c3a5e-640e-495c-9fc9-7316233f5215 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:32.366048 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:32.387231 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5676ba-d9d6-43f8-95a6-7891fb6125ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:32.389273 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5676ba-d9d6-43f8-95a6-7891fb6125ae None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:32.391953 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5676ba-d9d6-43f8-95a6-7891fb6125ae None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:32.392234 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5676ba-d9d6-43f8-95a6-7891fb6125ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:32.424031 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a3361b-ce35-46e6-ad82-222f2b209652 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:32.425522 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a3361b-ce35-46e6-ad82-222f2b209652 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:32.510390 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5676ba-d9d6-43f8-95a6-7891fb6125ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:32.511885 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5676ba-d9d6-43f8-95a6-7891fb6125ae None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:32.576373 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:32.599040 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 190/373] 10.222.0.61 () {58 vars in 1282 bytes} [Mon Aug 22 19:05:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 502 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 19:05:32.600798 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7b4e45-1157-4430-939c-336ccbe47f64 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:32.601084 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7b4e45-1157-4430-939c-336ccbe47f64 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:32.601358 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7b4e45-1157-4430-939c-336ccbe47f64 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:32.607792 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 191/374] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:05:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:05:32.607792 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7b4e45-1157-4430-939c-336ccbe47f64 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:32.659119 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 184/375] 10.222.0.61 () {58 vars in 1282 bytes} [Mon Aug 22 19:05:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 520 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 19:05:32.660240 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f0d658-1c02-417a-84b9-9937f9f55ff3 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:32.660579 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f0d658-1c02-417a-84b9-9937f9f55ff3 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:32.660784 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f0d658-1c02-417a-84b9-9937f9f55ff3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:32.683466 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-77f0d658-1c02-417a-84b9-9937f9f55ff3 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:05:32.684782 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-77f0d658-1c02-417a-84b9-9937f9f55ff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:05:32.709358 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:32.750053 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7b4e45-1157-4430-939c-336ccbe47f64 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:32.773355 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 185/376] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:05:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 19:05:32.774616 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f0d658-1c02-417a-84b9-9937f9f55ff3 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:32.790593 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de8093e-cf32-49bc-bc35-be5b8aaf2bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:32.791245 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de8093e-cf32-49bc-bc35-be5b8aaf2bb5 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:32.791525 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de8093e-cf32-49bc-bc35-be5b8aaf2bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:32.791843 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de8093e-cf32-49bc-bc35-be5b8aaf2bb5 None admin] PATH_INFO: `/v3/projects/47adea686aa0431fb9766ac68f4bf165` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:32.793148 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de8093e-cf32-49bc-bc35-be5b8aaf2bb5 None admin] RBAC: Authorizing `identity:get_project(project_id=47adea686aa0431fb9766ac68f4bf165)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:32.796065 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de8093e-cf32-49bc-bc35-be5b8aaf2bb5 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:32.797838 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 192/377] 10.222.0.61 () {56 vars in 1186 bytes} [Mon Aug 22 19:05:32 2022] GET /identity/v3/projects/47adea686aa0431fb9766ac68f4bf165 => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:32.809784 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de8093e-cf32-49bc-bc35-be5b8aaf2bb5 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:32.870584 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:32.906747 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f0d658-1c02-417a-84b9-9937f9f55ff3 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:32.944312 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f319cbf4-a93a-4896-bc5c-2b8f14aea2a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:32.944950 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f319cbf4-a93a-4896-bc5c-2b8f14aea2a3 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:32.945223 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f319cbf4-a93a-4896-bc5c-2b8f14aea2a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:32.945506 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f319cbf4-a93a-4896-bc5c-2b8f14aea2a3 None admin] PATH_INFO: `/v3/projects/4bc536d83f9841a59997d0d04e993cd8` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:32.946670 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f319cbf4-a93a-4896-bc5c-2b8f14aea2a3 None admin] RBAC: Authorizing `identity:get_project(project_id=4bc536d83f9841a59997d0d04e993cd8)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:32.948256 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f319cbf4-a93a-4896-bc5c-2b8f14aea2a3 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:32.950512 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 186/378] 10.222.0.61 () {56 vars in 1186 bytes} [Mon Aug 22 19:05:32 2022] GET /identity/v3/projects/4bc536d83f9841a59997d0d04e993cd8 => generated 344 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:32.956936 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f319cbf4-a93a-4896-bc5c-2b8f14aea2a3 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:32.975669 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de8093e-cf32-49bc-bc35-be5b8aaf2bb5 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:33.023238 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ef916d-b7eb-4b5e-8a59-80896a027dbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:33.023912 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ef916d-b7eb-4b5e-8a59-80896a027dbf None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:33.024191 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ef916d-b7eb-4b5e-8a59-80896a027dbf None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:33.024466 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ef916d-b7eb-4b5e-8a59-80896a027dbf None admin] PATH_INFO: `/v3/groups/f21c86cb093e4771896943da49969864` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:33.029261 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ef916d-b7eb-4b5e-8a59-80896a027dbf None admin] RBAC: Authorizing `identity:get_group(group_id=f21c86cb093e4771896943da49969864)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:33.030719 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ef916d-b7eb-4b5e-8a59-80896a027dbf None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:33.041950 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 193/379] 10.222.0.61 () {56 vars in 1180 bytes} [Mon Aug 22 19:05:32 2022] GET /identity/v3/groups/f21c86cb093e4771896943da49969864 => generated 226 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:33.049759 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ef916d-b7eb-4b5e-8a59-80896a027dbf None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:33.068673 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:33.197938 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f319cbf4-a93a-4896-bc5c-2b8f14aea2a3 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:33.233613 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ef916d-b7eb-4b5e-8a59-80896a027dbf None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:33.254633 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e645d3b7-4686-4f73-bfa8-308695ecb2d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:33.255269 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e645d3b7-4686-4f73-bfa8-308695ecb2d5 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:33.255542 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e645d3b7-4686-4f73-bfa8-308695ecb2d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:33.255836 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e645d3b7-4686-4f73-bfa8-308695ecb2d5 None admin] PATH_INFO: `/v3/groups/111ff86c43de4c45a0be30d3e1a0ea46` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:33.264256 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e645d3b7-4686-4f73-bfa8-308695ecb2d5 None admin] RBAC: Authorizing `identity:get_group(group_id=111ff86c43de4c45a0be30d3e1a0ea46)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:33.266250 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e645d3b7-4686-4f73-bfa8-308695ecb2d5 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:33.268202 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 187/380] 10.222.0.61 () {56 vars in 1180 bytes} [Mon Aug 22 19:05:32 2022] GET /identity/v3/groups/111ff86c43de4c45a0be30d3e1a0ea46 => generated 229 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 19:05:33.268966 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e645d3b7-4686-4f73-bfa8-308695ecb2d5 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:33.273305 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:33.293528 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b210924-deae-4316-86bd-320742b04083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:33.294825 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b210924-deae-4316-86bd-320742b04083 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:33.295305 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b210924-deae-4316-86bd-320742b04083 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:33.296363 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b210924-deae-4316-86bd-320742b04083 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:33.297616 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b210924-deae-4316-86bd-320742b04083 None admin] RBAC: Adding query filter params (group.id=f21c86cb093e4771896943da49969864, role.id=9f0952608a704dfca5fdbe4046331f21, scope.project.id=47adea686aa0431fb9766ac68f4bf165) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:33.298134 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b210924-deae-4316-86bd-320742b04083 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:33.300534 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b210924-deae-4316-86bd-320742b04083 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:33.311917 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 194/381] 10.222.0.61 () {56 vars in 1376 bytes} [Mon Aug 22 19:05:33 2022] GET /identity/v3/role_assignments?group.id=f21c86cb093e4771896943da49969864&scope.project.id=47adea686aa0431fb9766ac68f4bf165&role.id=9f0952608a704dfca5fdbe4046331f21 => generated 260 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 19:05:33.313768 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b210924-deae-4316-86bd-320742b04083 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:33.415556 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:33.447288 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b210924-deae-4316-86bd-320742b04083 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:33.483745 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e645d3b7-4686-4f73-bfa8-308695ecb2d5 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:33.491470 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3594f806-3d19-4331-82bb-eb38a16a0e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:33.492342 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3594f806-3d19-4331-82bb-eb38a16a0e46 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:33.492807 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3594f806-3d19-4331-82bb-eb38a16a0e46 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:33.493252 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3594f806-3d19-4331-82bb-eb38a16a0e46 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:33.501353 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3594f806-3d19-4331-82bb-eb38a16a0e46 None admin] RBAC: Adding query filter params (group.id=111ff86c43de4c45a0be30d3e1a0ea46, role.id=163c30c3b3c246beb849ce176f02d9da, scope.project.id=4bc536d83f9841a59997d0d04e993cd8) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:33.501942 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3594f806-3d19-4331-82bb-eb38a16a0e46 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:33.503712 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3594f806-3d19-4331-82bb-eb38a16a0e46 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:33.510263 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 195/382] 10.222.0.61 () {56 vars in 1376 bytes} [Mon Aug 22 19:05:33 2022] GET /identity/v3/role_assignments?group.id=111ff86c43de4c45a0be30d3e1a0ea46&scope.project.id=4bc536d83f9841a59997d0d04e993cd8&role.id=163c30c3b3c246beb849ce176f02d9da => generated 260 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:33.528978 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0642a1d1-3b4d-4ff7-a48f-31b21e627b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:33.529797 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0642a1d1-3b4d-4ff7-a48f-31b21e627b52 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:33.530138 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0642a1d1-3b4d-4ff7-a48f-31b21e627b52 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:33.530478 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0642a1d1-3b4d-4ff7-a48f-31b21e627b52 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:33.543606 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0642a1d1-3b4d-4ff7-a48f-31b21e627b52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 19:05:33.549563 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-0642a1d1-3b4d-4ff7-a48f-31b21e627b52 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 22 19:05:33.549563 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:05:33.549563 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:05:33.549563 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:05:33.549563 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:05:33.549563 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:05:33.549563 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:05:33.549563 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:05:33.549563 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:05:33.549563 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:05:33.549563 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:05:33.549563 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:05:33.549563 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 19:05:33.549563 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 19:05:33.549563 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 19:05:33.549563 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 19:05:33.550572 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 19:05:33.550572 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 19:05:33.550572 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 19:05:33.550572 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 19:05:33.550572 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:05:33.550572 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:05:33.550572 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:05:33.550572 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:05:33.550572 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:05:33.550572 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:05:33.550572 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:05:33.550572 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:05:33.550572 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:05:33.550572 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:05:33.550572 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:05:33.550572 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:05:33.551765 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:05:33.551765 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:05:33.551765 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:05:33.551765 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:05:33.551765 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 19:05:33.551765 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 19:05:33.551765 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 19:05:33.551765 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 19:05:33.551765 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 19:05:33.551765 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 19:05:33.551765 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 22 19:05:33.551765 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:05:33.551765 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 188/383] 10.222.0.61 () {56 vars in 1099 bytes} [Mon Aug 22 19:05:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:05:33.556822 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3594f806-3d19-4331-82bb-eb38a16a0e46 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:33.696202 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3594f806-3d19-4331-82bb-eb38a16a0e46 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:33.739796 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843aac97-5dc5-46e4-a7cf-854dde4f7d27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:33.740826 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843aac97-5dc5-46e4-a7cf-854dde4f7d27 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:33.741320 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843aac97-5dc5-46e4-a7cf-854dde4f7d27 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:33.741785 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843aac97-5dc5-46e4-a7cf-854dde4f7d27 None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:33.742645 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843aac97-5dc5-46e4-a7cf-854dde4f7d27 None admin] RBAC: Adding query filter params (name=member) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:33.743141 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843aac97-5dc5-46e4-a7cf-854dde4f7d27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:33.744733 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843aac97-5dc5-46e4-a7cf-854dde4f7d27 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:33.747438 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:33.756255 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 196/384] 10.222.0.61 () {56 vars in 1101 bytes} [Mon Aug 22 19:05:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:33.761807 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0642a1d1-3b4d-4ff7-a48f-31b21e627b52 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:33.828015 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:33.868514 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0642a1d1-3b4d-4ff7-a48f-31b21e627b52 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:33.904026 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0218ca-2e14-424e-9f90-abc9d8d80640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:33.904662 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0218ca-2e14-424e-9f90-abc9d8d80640 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:33.904935 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0218ca-2e14-424e-9f90-abc9d8d80640 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:33.905201 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0218ca-2e14-424e-9f90-abc9d8d80640 None admin] PATH_INFO: `/v3/groups/f21c86cb093e4771896943da49969864` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:33.906350 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0218ca-2e14-424e-9f90-abc9d8d80640 None admin] RBAC: Authorizing `identity:get_group(group_id=f21c86cb093e4771896943da49969864)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:33.907854 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0218ca-2e14-424e-9f90-abc9d8d80640 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:33.909783 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 189/385] 10.222.0.61 () {56 vars in 1180 bytes} [Mon Aug 22 19:05:33 2022] GET /identity/v3/groups/f21c86cb093e4771896943da49969864 => generated 226 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:33.916319 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843aac97-5dc5-46e4-a7cf-854dde4f7d27 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:34.033695 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843aac97-5dc5-46e4-a7cf-854dde4f7d27 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:34.069752 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d967dfc4-591f-4609-93d5-bf960cdae6ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:34.070432 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d967dfc4-591f-4609-93d5-bf960cdae6ab None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:34.070714 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d967dfc4-591f-4609-93d5-bf960cdae6ab None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:34.070980 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d967dfc4-591f-4609-93d5-bf960cdae6ab None admin] PATH_INFO: `/v3/projects/a4fe339e17324d15a22e51626818eecd` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:34.073318 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d967dfc4-591f-4609-93d5-bf960cdae6ab None admin] RBAC: Authorizing `identity:get_project(project_id=a4fe339e17324d15a22e51626818eecd)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:34.074759 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d967dfc4-591f-4609-93d5-bf960cdae6ab None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:34.076742 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 197/386] 10.222.0.61 () {56 vars in 1186 bytes} [Mon Aug 22 19:05:33 2022] GET /identity/v3/projects/a4fe339e17324d15a22e51626818eecd => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:34.084707 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0218ca-2e14-424e-9f90-abc9d8d80640 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:34.086824 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5870e168-6e58-498a-87d2-fcd394e3fe7e None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:34.087105 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5870e168-6e58-498a-87d2-fcd394e3fe7e None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:34.087374 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5870e168-6e58-498a-87d2-fcd394e3fe7e None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:34.089297 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 198/387] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:05:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:05:34.094609 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed1adc4-4965-43b8-bb37-4a6c5138b6cc None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:34.094894 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed1adc4-4965-43b8-bb37-4a6c5138b6cc None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:34.095171 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed1adc4-4965-43b8-bb37-4a6c5138b6cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:34.119458 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-1ed1adc4-4965-43b8-bb37-4a6c5138b6cc None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:05:34.120701 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-1ed1adc4-4965-43b8-bb37-4a6c5138b6cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:05:34.231101 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:34.252577 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0218ca-2e14-424e-9f90-abc9d8d80640 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:34.254263 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 199/388] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:05:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 19:05:34.294841 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:34.301921 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4932660a-d6d8-421e-b7c1-f32104bdef67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:34.302615 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4932660a-d6d8-421e-b7c1-f32104bdef67 None admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:34.303111 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4932660a-d6d8-421e-b7c1-f32104bdef67 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:34.303589 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4932660a-d6d8-421e-b7c1-f32104bdef67 None admin] PATH_INFO: `/v3/projects/a4fe339e17324d15a22e51626818eecd/groups/f21c86cb093e4771896943da49969864/roles/74a40bc819cb42eea7e966a4b2a016f1` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:34.305900 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4932660a-d6d8-421e-b7c1-f32104bdef67 None admin] RBAC: Authorizing `identity:create_grant(project_id=a4fe339e17324d15a22e51626818eecd, group_id=f21c86cb093e4771896943da49969864, role_id=74a40bc819cb42eea7e966a4b2a016f1)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:34.307348 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4932660a-d6d8-421e-b7c1-f32104bdef67 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:34.322192 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 190/389] 10.222.0.61 () {58 vars in 1442 bytes} [Mon Aug 22 19:05:34 2022] PUT /identity/v3/projects/a4fe339e17324d15a22e51626818eecd/groups/f21c86cb093e4771896943da49969864/roles/74a40bc819cb42eea7e966a4b2a016f1 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:05:34.558120 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be73a8c4-f730-4f96-b40d-b998370755a0 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:34.558614 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be73a8c4-f730-4f96-b40d-b998370755a0 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:34.558697 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be73a8c4-f730-4f96-b40d-b998370755a0 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:34.560291 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 200/390] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:05:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:05:34.568282 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896fe02a-aa2e-47d6-86a4-c10f9def5d12 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:34.568740 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896fe02a-aa2e-47d6-86a4-c10f9def5d12 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:34.568825 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896fe02a-aa2e-47d6-86a4-c10f9def5d12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:34.590830 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-896fe02a-aa2e-47d6-86a4-c10f9def5d12 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:05:34.592063 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-896fe02a-aa2e-47d6-86a4-c10f9def5d12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:05:34.638558 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be73a8c4-f730-4f96-b40d-b998370755a0 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:34.697637 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 191/391] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:05:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 19:05:34.764278 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be73a8c4-f730-4f96-b40d-b998370755a0 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:34.800714 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eddbbc5-b080-40ad-803f-fc0c31eb5cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:34.801413 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eddbbc5-b080-40ad-803f-fc0c31eb5cdd None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:34.801695 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eddbbc5-b080-40ad-803f-fc0c31eb5cdd None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:34.801966 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eddbbc5-b080-40ad-803f-fc0c31eb5cdd None admin] PATH_INFO: `/v3/roles/member` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:34.813666 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eddbbc5-b080-40ad-803f-fc0c31eb5cdd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 19:05:34.818438 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-1eddbbc5-b080-40ad-803f-fc0c31eb5cdd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 22 19:05:34.818438 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:05:34.818438 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:05:34.818438 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:05:34.818438 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:05:34.818438 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:05:34.818438 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:05:34.818438 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:05:34.818438 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:05:34.818438 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:05:34.818438 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:05:34.818438 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:05:34.818438 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 19:05:34.818438 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 19:05:34.818438 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 19:05:34.818438 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 19:05:34.819323 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 19:05:34.819323 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 19:05:34.819323 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 19:05:34.819323 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 19:05:34.819323 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:05:34.819323 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:05:34.819323 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:05:34.819323 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:05:34.819323 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:05:34.819323 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:05:34.819323 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:05:34.819323 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:05:34.819323 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:05:34.819323 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:05:34.819323 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:05:34.819323 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:05:34.820143 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:05:34.820143 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:05:34.820143 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:05:34.820143 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:05:34.820143 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 19:05:34.820143 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 19:05:34.820143 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 19:05:34.820143 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 19:05:34.820143 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 19:05:34.820143 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 19:05:34.820143 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 22 19:05:34.820143 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:05:34.820143 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 201/392] 10.222.0.61 () {56 vars in 1099 bytes} [Mon Aug 22 19:05:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:05:34.827633 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896fe02a-aa2e-47d6-86a4-c10f9def5d12 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:34.953691 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896fe02a-aa2e-47d6-86a4-c10f9def5d12 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:34.983464 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:34.996313 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c06c03e-f07d-49c6-91a6-d393c8d4aa9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:34.997027 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c06c03e-f07d-49c6-91a6-d393c8d4aa9c None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:34.997303 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c06c03e-f07d-49c6-91a6-d393c8d4aa9c None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:34.997629 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c06c03e-f07d-49c6-91a6-d393c8d4aa9c None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:34.998331 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c06c03e-f07d-49c6-91a6-d393c8d4aa9c None admin] RBAC: Adding query filter params (name=member) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:34.998634 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c06c03e-f07d-49c6-91a6-d393c8d4aa9c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:35.000633 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c06c03e-f07d-49c6-91a6-d393c8d4aa9c None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:35.016741 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 192/393] 10.222.0.61 () {56 vars in 1101 bytes} [Mon Aug 22 19:05:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:35.018075 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eddbbc5-b080-40ad-803f-fc0c31eb5cdd None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:35.168773 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:35.264901 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eddbbc5-b080-40ad-803f-fc0c31eb5cdd None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:35.293431 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c06c03e-f07d-49c6-91a6-d393c8d4aa9c None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:35.340396 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb45ffb-ba9f-4724-8d13-3236fbce4efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:35.341062 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb45ffb-ba9f-4724-8d13-3236fbce4efe None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:35.341348 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb45ffb-ba9f-4724-8d13-3236fbce4efe None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:35.341626 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb45ffb-ba9f-4724-8d13-3236fbce4efe None admin] PATH_INFO: `/v3/groups/f21c86cb093e4771896943da49969864` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:35.346385 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb45ffb-ba9f-4724-8d13-3236fbce4efe None admin] RBAC: Authorizing `identity:get_group(group_id=f21c86cb093e4771896943da49969864)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:35.347857 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb45ffb-ba9f-4724-8d13-3236fbce4efe None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:35.349778 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 202/394] 10.222.0.61 () {56 vars in 1180 bytes} [Mon Aug 22 19:05:35 2022] GET /identity/v3/groups/f21c86cb093e4771896943da49969864 => generated 226 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:35.356721 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb45ffb-ba9f-4724-8d13-3236fbce4efe None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:35.426990 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c06c03e-f07d-49c6-91a6-d393c8d4aa9c None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:35.496495 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83a21fe-c361-49d3-aab8-bfa446db4bb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:35.497131 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83a21fe-c361-49d3-aab8-bfa446db4bb1 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:35.497455 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83a21fe-c361-49d3-aab8-bfa446db4bb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:35.497781 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83a21fe-c361-49d3-aab8-bfa446db4bb1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:35.516959 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83a21fe-c361-49d3-aab8-bfa446db4bb1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 19:05:35.524638 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-e83a21fe-c361-49d3-aab8-bfa446db4bb1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 19:05:35.524638 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:05:35.524638 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:05:35.524638 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:05:35.524638 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:05:35.524638 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:05:35.524638 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:05:35.524638 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:05:35.524638 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:05:35.524638 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:05:35.524638 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:05:35.524638 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:05:35.524638 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 19:05:35.524638 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 19:05:35.524638 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 19:05:35.524638 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 19:05:35.525521 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 19:05:35.525521 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 19:05:35.525521 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 19:05:35.525521 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 19:05:35.525521 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:05:35.525521 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:05:35.525521 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:05:35.525521 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:05:35.525521 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:05:35.525521 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:05:35.525521 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:05:35.525521 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:05:35.525521 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:05:35.525521 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:05:35.525521 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:05:35.525521 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:05:35.526342 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:05:35.526342 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:05:35.526342 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:05:35.526342 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:05:35.526342 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 19:05:35.526342 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 19:05:35.526342 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 19:05:35.526342 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 19:05:35.526342 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 19:05:35.526342 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 19:05:35.526342 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 19:05:35.526342 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:05:35.527403 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 193/395] 10.222.0.61 () {56 vars in 1114 bytes} [Mon Aug 22 19:05:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 19:05:35.536573 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83a21fe-c361-49d3-aab8-bfa446db4bb1 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:35.585377 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb45ffb-ba9f-4724-8d13-3236fbce4efe None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:35.617314 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:35.661495 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:35.694616 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83a21fe-c361-49d3-aab8-bfa446db4bb1 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:35.711284 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b1e329-7904-4957-901c-9e31e78103b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:35.711975 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b1e329-7904-4957-901c-9e31e78103b6 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:35.712261 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b1e329-7904-4957-901c-9e31e78103b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:35.712536 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b1e329-7904-4957-901c-9e31e78103b6 None admin] PATH_INFO: `/v3/projects/47adea686aa0431fb9766ac68f4bf165` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:35.713837 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b1e329-7904-4957-901c-9e31e78103b6 None admin] RBAC: Authorizing `identity:get_project(project_id=47adea686aa0431fb9766ac68f4bf165)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:35.715311 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b1e329-7904-4957-901c-9e31e78103b6 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:35.725451 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 203/396] 10.222.0.61 () {56 vars in 1186 bytes} [Mon Aug 22 19:05:35 2022] GET /identity/v3/projects/47adea686aa0431fb9766ac68f4bf165 => generated 302 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:35.735297 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b1e329-7904-4957-901c-9e31e78103b6 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:35.750807 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5e1ed7-b5d0-4e51-8aa0-08747dade1ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:35.752803 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5e1ed7-b5d0-4e51-8aa0-08747dade1ef None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:35.753899 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5e1ed7-b5d0-4e51-8aa0-08747dade1ef None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:35.754485 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5e1ed7-b5d0-4e51-8aa0-08747dade1ef None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:35.755431 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5e1ed7-b5d0-4e51-8aa0-08747dade1ef None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:35.756501 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5e1ed7-b5d0-4e51-8aa0-08747dade1ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:35.758209 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5e1ed7-b5d0-4e51-8aa0-08747dade1ef None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:35.778195 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 194/397] 10.222.0.61 () {56 vars in 1111 bytes} [Mon Aug 22 19:05:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:35.787560 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5e1ed7-b5d0-4e51-8aa0-08747dade1ef None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:35.943898 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b1e329-7904-4957-901c-9e31e78103b6 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:35.979458 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f8f716-0d6c-414a-bcf1-7e5fc59f3903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:35.980459 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f8f716-0d6c-414a-bcf1-7e5fc59f3903 None admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:35.980948 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f8f716-0d6c-414a-bcf1-7e5fc59f3903 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:35.981420 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f8f716-0d6c-414a-bcf1-7e5fc59f3903 None admin] PATH_INFO: `/v3/projects/47adea686aa0431fb9766ac68f4bf165/groups/f21c86cb093e4771896943da49969864/roles/74a40bc819cb42eea7e966a4b2a016f1` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:35.983556 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f8f716-0d6c-414a-bcf1-7e5fc59f3903 None admin] RBAC: Authorizing `identity:create_grant(project_id=47adea686aa0431fb9766ac68f4bf165, group_id=f21c86cb093e4771896943da49969864, role_id=74a40bc819cb42eea7e966a4b2a016f1)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:35.985287 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f8f716-0d6c-414a-bcf1-7e5fc59f3903 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:36.004222 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 204/398] 10.222.0.61 () {58 vars in 1442 bytes} [Mon Aug 22 19:05:35 2022] PUT /identity/v3/projects/47adea686aa0431fb9766ac68f4bf165/groups/f21c86cb093e4771896943da49969864/roles/74a40bc819cb42eea7e966a4b2a016f1 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:05:36.009746 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5e1ed7-b5d0-4e51-8aa0-08747dade1ef None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:36.023662 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:36.066977 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff1c4a8-5c78-4671-9772-647ff267a348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:36.067629 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff1c4a8-5c78-4671-9772-647ff267a348 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:36.067927 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff1c4a8-5c78-4671-9772-647ff267a348 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:36.068201 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff1c4a8-5c78-4671-9772-647ff267a348 None admin] PATH_INFO: `/v3/groups/f21c86cb093e4771896943da49969864` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:36.072510 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff1c4a8-5c78-4671-9772-647ff267a348 None admin] RBAC: Authorizing `identity:get_group(group_id=f21c86cb093e4771896943da49969864)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:36.074089 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff1c4a8-5c78-4671-9772-647ff267a348 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:36.076157 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 195/399] 10.222.0.61 () {56 vars in 1180 bytes} [Mon Aug 22 19:05:35 2022] GET /identity/v3/groups/f21c86cb093e4771896943da49969864 => generated 226 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:36.082717 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f8f716-0d6c-414a-bcf1-7e5fc59f3903 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:36.158288 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:36.175150 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f137b936-7cb9-4af0-80a3-caaa9e7a753a None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:36.175462 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f137b936-7cb9-4af0-80a3-caaa9e7a753a None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:36.175730 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f137b936-7cb9-4af0-80a3-caaa9e7a753a None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:36.177076 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 196/400] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:05:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:05:36.184609 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703a3e1a-c04e-4e25-8362-005e945dd092 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:36.185125 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703a3e1a-c04e-4e25-8362-005e945dd092 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:36.185571 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703a3e1a-c04e-4e25-8362-005e945dd092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:36.204065 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f8f716-0d6c-414a-bcf1-7e5fc59f3903 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:36.214940 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-703a3e1a-c04e-4e25-8362-005e945dd092 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:05:36.216177 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-703a3e1a-c04e-4e25-8362-005e945dd092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:05:36.242057 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e46d136-02e7-4a34-b4a6-c722ff4eabd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:36.242630 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e46d136-02e7-4a34-b4a6-c722ff4eabd6 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:36.242912 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e46d136-02e7-4a34-b4a6-c722ff4eabd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:36.243186 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e46d136-02e7-4a34-b4a6-c722ff4eabd6 None admin] PATH_INFO: `/v3/projects/a4fe339e17324d15a22e51626818eecd` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:36.244378 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e46d136-02e7-4a34-b4a6-c722ff4eabd6 None admin] RBAC: Authorizing `identity:get_project(project_id=a4fe339e17324d15a22e51626818eecd)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:36.247172 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e46d136-02e7-4a34-b4a6-c722ff4eabd6 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:36.247845 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 205/401] 10.222.0.61 () {56 vars in 1186 bytes} [Mon Aug 22 19:05:36 2022] GET /identity/v3/projects/a4fe339e17324d15a22e51626818eecd => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:36.260631 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e46d136-02e7-4a34-b4a6-c722ff4eabd6 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:36.315678 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 197/402] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:05:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 19:05:36.316218 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825dd3e0-81cb-468c-a617-b007c92219bb None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:36.316218 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825dd3e0-81cb-468c-a617-b007c92219bb None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:36.316475 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825dd3e0-81cb-468c-a617-b007c92219bb None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:36.318076 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 198/403] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:05:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:05:36.323515 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b90a82-7f83-42c5-9db0-9bde47f591f4 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:36.323837 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b90a82-7f83-42c5-9db0-9bde47f591f4 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:36.324113 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b90a82-7f83-42c5-9db0-9bde47f591f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:36.343546 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-f9b90a82-7f83-42c5-9db0-9bde47f591f4 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:05:36.345330 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-f9b90a82-7f83-42c5-9db0-9bde47f591f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:05:36.385819 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e46d136-02e7-4a34-b4a6-c722ff4eabd6 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:36.429939 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:36.459781 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 199/404] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:05:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 19:05:36.480920 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3b58da-ca7b-4fcb-8b8b-2e27bf18b5f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:36.481609 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3b58da-ca7b-4fcb-8b8b-2e27bf18b5f3 None admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:36.481889 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3b58da-ca7b-4fcb-8b8b-2e27bf18b5f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:36.482160 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3b58da-ca7b-4fcb-8b8b-2e27bf18b5f3 None admin] PATH_INFO: `/v3/projects/a4fe339e17324d15a22e51626818eecd/groups/f21c86cb093e4771896943da49969864/roles/9f0952608a704dfca5fdbe4046331f21` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:36.489663 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3b58da-ca7b-4fcb-8b8b-2e27bf18b5f3 None admin] RBAC: Authorizing `identity:create_grant(project_id=a4fe339e17324d15a22e51626818eecd, group_id=f21c86cb093e4771896943da49969864, role_id=9f0952608a704dfca5fdbe4046331f21)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:36.491107 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3b58da-ca7b-4fcb-8b8b-2e27bf18b5f3 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:36.524241 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 206/405] 10.222.0.61 () {58 vars in 1442 bytes} [Mon Aug 22 19:05:36 2022] PUT /identity/v3/projects/a4fe339e17324d15a22e51626818eecd/groups/f21c86cb093e4771896943da49969864/roles/9f0952608a704dfca5fdbe4046331f21 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:05:36.624432 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b90a82-7f83-42c5-9db0-9bde47f591f4 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:36.676617 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:36.757439 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b90a82-7f83-42c5-9db0-9bde47f591f4 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:36.795716 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c3782a-cc3e-4db3-876f-4e0c27a4540d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:36.796282 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c3782a-cc3e-4db3-876f-4e0c27a4540d None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:36.796555 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c3782a-cc3e-4db3-876f-4e0c27a4540d None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:36.796867 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c3782a-cc3e-4db3-876f-4e0c27a4540d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:36.801845 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3b58da-ca7b-4fcb-8b8b-2e27bf18b5f3 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:36.808956 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c3782a-cc3e-4db3-876f-4e0c27a4540d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 19:05:36.814169 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-f0c3782a-cc3e-4db3-876f-4e0c27a4540d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 19:05:36.814169 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:05:36.814169 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:05:36.814169 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:05:36.814169 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:05:36.814169 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:05:36.814169 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:05:36.814169 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:05:36.814169 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:05:36.814169 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:05:36.814169 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:05:36.814169 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:05:36.814169 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 19:05:36.814169 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 19:05:36.814169 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 19:05:36.814169 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 19:05:36.815096 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 19:05:36.815096 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 19:05:36.815096 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 19:05:36.815096 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 19:05:36.815096 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:05:36.815096 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:05:36.815096 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:05:36.815096 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:05:36.815096 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:05:36.815096 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:05:36.815096 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:05:36.815096 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:05:36.815096 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:05:36.815096 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:05:36.815096 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:05:36.815096 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:05:36.815967 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:05:36.815967 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:05:36.815967 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:05:36.815967 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:05:36.815967 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 19:05:36.815967 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 19:05:36.815967 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 19:05:36.815967 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 19:05:36.815967 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 19:05:36.815967 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 19:05:36.815967 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 19:05:36.815967 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:05:36.815967 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 200/406] 10.222.0.61 () {56 vars in 1114 bytes} [Mon Aug 22 19:05:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:05:36.821306 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c3782a-cc3e-4db3-876f-4e0c27a4540d None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:36.925359 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3b58da-ca7b-4fcb-8b8b-2e27bf18b5f3 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:36.936060 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:36.944359 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c3782a-cc3e-4db3-876f-4e0c27a4540d None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:36.964620 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25857dc-02de-4746-9d2c-34f6014ea48b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:36.965257 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25857dc-02de-4746-9d2c-34f6014ea48b None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:36.965537 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25857dc-02de-4746-9d2c-34f6014ea48b None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:36.965808 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25857dc-02de-4746-9d2c-34f6014ea48b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:36.980659 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25857dc-02de-4746-9d2c-34f6014ea48b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 19:05:36.985547 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf224da6-8cc2-4a40-988c-adae1a013529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:36.986503 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf224da6-8cc2-4a40-988c-adae1a013529 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:36.986899 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-e25857dc-02de-4746-9d2c-34f6014ea48b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 19:05:36.986899 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:05:36.986899 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:05:36.986899 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:05:36.986899 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:05:36.986899 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:05:36.986899 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:05:36.986899 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:05:36.986899 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:05:36.986899 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:05:36.986899 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:05:36.986899 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:05:36.986899 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 19:05:36.986899 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 19:05:36.986899 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 19:05:36.986899 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 19:05:36.987853 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 19:05:36.987853 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 19:05:36.987853 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 19:05:36.987853 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 19:05:36.987853 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:05:36.987853 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:05:36.987853 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:05:36.987853 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:05:36.987853 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:05:36.987853 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:05:36.987853 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:05:36.987853 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:05:36.987853 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:05:36.987853 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:05:36.987853 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:05:36.987853 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:05:36.988682 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:05:36.988682 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:05:36.988682 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:05:36.988682 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:05:36.988682 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 19:05:36.988682 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 19:05:36.988682 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 19:05:36.988682 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 19:05:36.988682 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 19:05:36.988682 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 19:05:36.988682 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 19:05:36.988682 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:05:36.988682 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 207/407] 10.222.0.61 () {56 vars in 1096 bytes} [Mon Aug 22 19:05:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:05:36.990108 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51fa9eb-78de-4752-9e95-8d3098907343 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:36.990206 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51fa9eb-78de-4752-9e95-8d3098907343 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:36.990434 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51fa9eb-78de-4752-9e95-8d3098907343 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:36.990718 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf224da6-8cc2-4a40-988c-adae1a013529 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:36.991197 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf224da6-8cc2-4a40-988c-adae1a013529 None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:36.991798 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 208/408] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:05:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:05:36.992564 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf224da6-8cc2-4a40-988c-adae1a013529 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:36.993292 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf224da6-8cc2-4a40-988c-adae1a013529 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:36.994897 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf224da6-8cc2-4a40-988c-adae1a013529 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:36.996487 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51fa9eb-78de-4752-9e95-8d3098907343 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:37.006097 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 201/409] 10.222.0.61 () {56 vars in 1111 bytes} [Mon Aug 22 19:05:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:37.007703 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8902f0-9adb-4f87-8b50-456d38a2f13b None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:37.008183 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8902f0-9adb-4f87-8b50-456d38a2f13b None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:37.008663 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8902f0-9adb-4f87-8b50-456d38a2f13b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:37.027478 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-9f8902f0-9adb-4f87-8b50-456d38a2f13b None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:05:37.029061 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-9f8902f0-9adb-4f87-8b50-456d38a2f13b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:05:37.060207 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:37.119284 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 202/410] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:05:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 19:05:37.120718 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8902f0-9adb-4f87-8b50-456d38a2f13b None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:37.160546 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51fa9eb-78de-4752-9e95-8d3098907343 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:37.215245 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1d3858-8984-4bdf-8fb0-2224c38c4f99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:37.216263 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1d3858-8984-4bdf-8fb0-2224c38c4f99 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:37.216756 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1d3858-8984-4bdf-8fb0-2224c38c4f99 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:37.217341 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1d3858-8984-4bdf-8fb0-2224c38c4f99 None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:37.218291 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1d3858-8984-4bdf-8fb0-2224c38c4f99 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:37.218813 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1d3858-8984-4bdf-8fb0-2224c38c4f99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:37.220422 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1d3858-8984-4bdf-8fb0-2224c38c4f99 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:37.230470 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 209/411] 10.222.0.61 () {56 vars in 1099 bytes} [Mon Aug 22 19:05:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:37.236091 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1d3858-8984-4bdf-8fb0-2224c38c4f99 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:37.313691 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:37.330895 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8902f0-9adb-4f87-8b50-456d38a2f13b None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:37.379381 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1d3858-8984-4bdf-8fb0-2224c38c4f99 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:37.384377 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:37.389252 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e9291d-02cd-4133-9232-b289f862b288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:37.391294 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e9291d-02cd-4133-9232-b289f862b288 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:37.394418 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e9291d-02cd-4133-9232-b289f862b288 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:37.394928 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e9291d-02cd-4133-9232-b289f862b288 None admin] PATH_INFO: `/v3/groups/f21c86cb093e4771896943da49969864` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:37.396288 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e9291d-02cd-4133-9232-b289f862b288 None admin] RBAC: Authorizing `identity:get_group(group_id=f21c86cb093e4771896943da49969864)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:37.397919 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e9291d-02cd-4133-9232-b289f862b288 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:37.402609 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 203/412] 10.222.0.61 () {56 vars in 1180 bytes} [Mon Aug 22 19:05:37 2022] GET /identity/v3/groups/f21c86cb093e4771896943da49969864 => generated 226 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:37.406654 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e9291d-02cd-4133-9232-b289f862b288 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:37.449750 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c37ad8-5666-4d52-a61f-fffac2e4b4c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:37.450388 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c37ad8-5666-4d52-a61f-fffac2e4b4c1 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:37.450664 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c37ad8-5666-4d52-a61f-fffac2e4b4c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:37.450934 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c37ad8-5666-4d52-a61f-fffac2e4b4c1 None admin] PATH_INFO: `/v3/groups/111ff86c43de4c45a0be30d3e1a0ea46` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:37.452093 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c37ad8-5666-4d52-a61f-fffac2e4b4c1 None admin] RBAC: Authorizing `identity:get_group(group_id=111ff86c43de4c45a0be30d3e1a0ea46)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:37.453607 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c37ad8-5666-4d52-a61f-fffac2e4b4c1 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:37.455249 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 210/413] 10.222.0.61 () {56 vars in 1180 bytes} [Mon Aug 22 19:05:37 2022] GET /identity/v3/groups/111ff86c43de4c45a0be30d3e1a0ea46 => generated 229 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:37.460005 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c37ad8-5666-4d52-a61f-fffac2e4b4c1 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:37.533059 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e9291d-02cd-4133-9232-b289f862b288 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:37.561281 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c37ad8-5666-4d52-a61f-fffac2e4b4c1 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:37.571905 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a36d840-71b1-46c0-ac3e-4f79e3968d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:37.572933 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a36d840-71b1-46c0-ac3e-4f79e3968d3e None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:37.573413 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a36d840-71b1-46c0-ac3e-4f79e3968d3e None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:37.573876 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a36d840-71b1-46c0-ac3e-4f79e3968d3e None admin] PATH_INFO: `/v3/projects/47adea686aa0431fb9766ac68f4bf165` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:37.575308 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a36d840-71b1-46c0-ac3e-4f79e3968d3e None admin] RBAC: Authorizing `identity:get_project(project_id=47adea686aa0431fb9766ac68f4bf165)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:37.577069 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a36d840-71b1-46c0-ac3e-4f79e3968d3e None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:37.579520 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 204/414] 10.222.0.61 () {56 vars in 1186 bytes} [Mon Aug 22 19:05:37 2022] GET /identity/v3/projects/47adea686aa0431fb9766ac68f4bf165 => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 19:05:37.580500 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a36d840-71b1-46c0-ac3e-4f79e3968d3e None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:37.596762 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7bef1e-7185-4da2-b67e-981a260fe77e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:37.597482 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7bef1e-7185-4da2-b67e-981a260fe77e None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:37.597696 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7bef1e-7185-4da2-b67e-981a260fe77e None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:37.597932 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7bef1e-7185-4da2-b67e-981a260fe77e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:37.608387 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7bef1e-7185-4da2-b67e-981a260fe77e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 19:05:37.608956 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:37.613725 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-6c7bef1e-7185-4da2-b67e-981a260fe77e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 22 19:05:37.613725 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:05:37.613725 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:05:37.613725 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:05:37.613725 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:05:37.613725 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:05:37.613725 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:05:37.613725 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:05:37.613725 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:05:37.613725 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:05:37.613725 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:05:37.613725 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:05:37.613725 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 19:05:37.613725 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 19:05:37.613725 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 19:05:37.613725 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 19:05:37.616027 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 19:05:37.616027 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 19:05:37.616027 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 19:05:37.616027 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 19:05:37.616027 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:05:37.616027 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:05:37.616027 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:05:37.616027 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:05:37.616027 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:05:37.616027 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:05:37.616027 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:05:37.616027 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:05:37.616027 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:05:37.616027 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:05:37.616027 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:05:37.616027 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:05:37.616841 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:05:37.616841 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:05:37.616841 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:05:37.616841 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:05:37.616841 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 19:05:37.616841 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 19:05:37.616841 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 19:05:37.616841 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 19:05:37.616841 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 19:05:37.616841 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 19:05:37.616841 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 22 19:05:37.616841 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:05:37.616841 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 211/415] 10.222.0.61 () {56 vars in 1099 bytes} [Mon Aug 22 19:05:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:05:37.621208 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7bef1e-7185-4da2-b67e-981a260fe77e None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:37.652820 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:37.700670 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a36d840-71b1-46c0-ac3e-4f79e3968d3e None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:37.730938 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7bef1e-7185-4da2-b67e-981a260fe77e None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:37.740400 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75810e6-c0cd-4a35-8300-9421a9733d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:37.741230 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75810e6-c0cd-4a35-8300-9421a9733d0b None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:37.741738 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75810e6-c0cd-4a35-8300-9421a9733d0b None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:37.742169 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75810e6-c0cd-4a35-8300-9421a9733d0b None admin] PATH_INFO: `/v3/projects/4bc536d83f9841a59997d0d04e993cd8` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:37.743530 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75810e6-c0cd-4a35-8300-9421a9733d0b None admin] RBAC: Authorizing `identity:get_project(project_id=4bc536d83f9841a59997d0d04e993cd8)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:37.745173 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75810e6-c0cd-4a35-8300-9421a9733d0b None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:37.747506 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 205/416] 10.222.0.61 () {56 vars in 1186 bytes} [Mon Aug 22 19:05:37 2022] GET /identity/v3/projects/4bc536d83f9841a59997d0d04e993cd8 => generated 344 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:37.748427 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75810e6-c0cd-4a35-8300-9421a9733d0b None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:37.773162 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e90ca9a-ee6f-4333-9920-d95ad8f63d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:37.774496 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e90ca9a-ee6f-4333-9920-d95ad8f63d09 None admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:37.774496 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e90ca9a-ee6f-4333-9920-d95ad8f63d09 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:37.774734 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e90ca9a-ee6f-4333-9920-d95ad8f63d09 None admin] PATH_INFO: `/v3/projects/47adea686aa0431fb9766ac68f4bf165/groups/f21c86cb093e4771896943da49969864/roles/9f0952608a704dfca5fdbe4046331f21` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:37.777266 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e90ca9a-ee6f-4333-9920-d95ad8f63d09 None admin] RBAC: Authorizing `identity:create_grant(project_id=47adea686aa0431fb9766ac68f4bf165, group_id=f21c86cb093e4771896943da49969864, role_id=9f0952608a704dfca5fdbe4046331f21)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:37.778891 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e90ca9a-ee6f-4333-9920-d95ad8f63d09 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:37.794545 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 212/417] 10.222.0.61 () {58 vars in 1442 bytes} [Mon Aug 22 19:05:37 2022] PUT /identity/v3/projects/47adea686aa0431fb9766ac68f4bf165/groups/f21c86cb093e4771896943da49969864/roles/9f0952608a704dfca5fdbe4046331f21 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:05:37.795494 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e90ca9a-ee6f-4333-9920-d95ad8f63d09 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:37.861678 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:37.873459 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75810e6-c0cd-4a35-8300-9421a9733d0b None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:37.897395 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:37.907907 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e90ca9a-ee6f-4333-9920-d95ad8f63d09 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:37.918649 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080420c0-0b50-473b-8e59-d6a97c2bed1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:37.919500 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080420c0-0b50-473b-8e59-d6a97c2bed1e None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:37.920006 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080420c0-0b50-473b-8e59-d6a97c2bed1e None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:37.920586 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080420c0-0b50-473b-8e59-d6a97c2bed1e None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:37.921504 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080420c0-0b50-473b-8e59-d6a97c2bed1e None admin] RBAC: Adding query filter params (name=member) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:37.922633 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080420c0-0b50-473b-8e59-d6a97c2bed1e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:37.924201 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080420c0-0b50-473b-8e59-d6a97c2bed1e None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:37.934179 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 206/418] 10.222.0.61 () {56 vars in 1101 bytes} [Mon Aug 22 19:05:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:37.944836 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080420c0-0b50-473b-8e59-d6a97c2bed1e None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:37.971954 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f0663d-4b5e-4156-8773-6d854ee53d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:37.972642 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f0663d-4b5e-4156-8773-6d854ee53d93 None admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:37.972918 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f0663d-4b5e-4156-8773-6d854ee53d93 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:37.973189 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f0663d-4b5e-4156-8773-6d854ee53d93 None admin] PATH_INFO: `/v3/projects/4bc536d83f9841a59997d0d04e993cd8/groups/111ff86c43de4c45a0be30d3e1a0ea46/roles/163c30c3b3c246beb849ce176f02d9da` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:37.978489 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f0663d-4b5e-4156-8773-6d854ee53d93 None admin] RBAC: Authorizing `identity:create_grant(project_id=4bc536d83f9841a59997d0d04e993cd8, group_id=111ff86c43de4c45a0be30d3e1a0ea46, role_id=163c30c3b3c246beb849ce176f02d9da)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:37.981345 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f0663d-4b5e-4156-8773-6d854ee53d93 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:38.007398 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 213/419] 10.222.0.61 () {58 vars in 1442 bytes} [Mon Aug 22 19:05:37 2022] PUT /identity/v3/projects/4bc536d83f9841a59997d0d04e993cd8/groups/111ff86c43de4c45a0be30d3e1a0ea46/roles/163c30c3b3c246beb849ce176f02d9da => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:05:38.089080 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080420c0-0b50-473b-8e59-d6a97c2bed1e None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:38.127443 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0a82c3-e1fd-4c56-aa2f-ffbcf2e9bcc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:38.128078 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0a82c3-e1fd-4c56-aa2f-ffbcf2e9bcc2 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:38.128362 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0a82c3-e1fd-4c56-aa2f-ffbcf2e9bcc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:38.128640 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0a82c3-e1fd-4c56-aa2f-ffbcf2e9bcc2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:38.177184 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:38.206332 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0a82c3-e1fd-4c56-aa2f-ffbcf2e9bcc2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:38.207842 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0a82c3-e1fd-4c56-aa2f-ffbcf2e9bcc2 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:38.312241 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 207/420] 10.222.0.61 () {58 vars in 1282 bytes} [Mon Aug 22 19:05:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 19:05:38.315087 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ad9435-4e68-4de1-a650-38cb99795fbe None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:38.315485 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ad9435-4e68-4de1-a650-38cb99795fbe None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:38.315895 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ad9435-4e68-4de1-a650-38cb99795fbe None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:38.317505 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0a82c3-e1fd-4c56-aa2f-ffbcf2e9bcc2 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:38.317964 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 214/421] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:05:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 19:05:38.326037 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382b11a9-85b2-411c-85f6-fe14b92024d4 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:38.331329 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382b11a9-85b2-411c-85f6-fe14b92024d4 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:38.331791 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382b11a9-85b2-411c-85f6-fe14b92024d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:38.361230 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-382b11a9-85b2-411c-85f6-fe14b92024d4 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:05:38.363612 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-382b11a9-85b2-411c-85f6-fe14b92024d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:05:38.415871 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:38.481206 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 215/422] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:05:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 19:05:38.497602 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0a82c3-e1fd-4c56-aa2f-ffbcf2e9bcc2 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:38.539574 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ae2093-2ed3-4569-bbd9-b576ef8b7912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:38.540493 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ae2093-2ed3-4569-bbd9-b576ef8b7912 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:38.540921 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ae2093-2ed3-4569-bbd9-b576ef8b7912 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:38.541339 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ae2093-2ed3-4569-bbd9-b576ef8b7912 None admin] PATH_INFO: `/v3/projects/a4fe339e17324d15a22e51626818eecd` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:38.542598 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ae2093-2ed3-4569-bbd9-b576ef8b7912 None admin] RBAC: Authorizing `identity:get_project(project_id=a4fe339e17324d15a22e51626818eecd)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:38.544216 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ae2093-2ed3-4569-bbd9-b576ef8b7912 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:38.546292 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 208/423] 10.222.0.61 () {56 vars in 1186 bytes} [Mon Aug 22 19:05:38 2022] GET /identity/v3/projects/a4fe339e17324d15a22e51626818eecd => generated 298 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 19:05:38.552776 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382b11a9-85b2-411c-85f6-fe14b92024d4 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:38.638724 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:38.651952 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a531f060-969b-4160-b3d4-022bff55f85c None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:38.652831 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a531f060-969b-4160-b3d4-022bff55f85c None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:38.653110 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a531f060-969b-4160-b3d4-022bff55f85c None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:38.655022 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 209/424] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:05:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:05:38.662138 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ad23ba-7daa-4227-a757-ba884b5ac9d2 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:38.662421 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ad23ba-7daa-4227-a757-ba884b5ac9d2 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:38.662690 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ad23ba-7daa-4227-a757-ba884b5ac9d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:38.667893 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382b11a9-85b2-411c-85f6-fe14b92024d4 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:38.687857 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-d0ad23ba-7daa-4227-a757-ba884b5ac9d2 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:05:38.687857 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-d0ad23ba-7daa-4227-a757-ba884b5ac9d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:05:38.729239 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e7eb7e-9243-4191-8395-7d6f91faddcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:38.729877 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e7eb7e-9243-4191-8395-7d6f91faddcd None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:38.730151 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e7eb7e-9243-4191-8395-7d6f91faddcd None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:38.730420 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e7eb7e-9243-4191-8395-7d6f91faddcd None admin] PATH_INFO: `/v3/groups/f21c86cb093e4771896943da49969864` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:38.732362 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e7eb7e-9243-4191-8395-7d6f91faddcd None admin] RBAC: Authorizing `identity:get_group(group_id=f21c86cb093e4771896943da49969864)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:38.733807 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e7eb7e-9243-4191-8395-7d6f91faddcd None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:38.735413 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 216/425] 10.222.0.61 () {56 vars in 1180 bytes} [Mon Aug 22 19:05:38 2022] GET /identity/v3/groups/f21c86cb093e4771896943da49969864 => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:38.741659 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e7eb7e-9243-4191-8395-7d6f91faddcd None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:38.804903 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 210/426] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:05:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 19:05:38.860083 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e7eb7e-9243-4191-8395-7d6f91faddcd None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:38.896083 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d506517c-e579-45ab-8cd2-b1f32f37fed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:38.896843 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d506517c-e579-45ab-8cd2-b1f32f37fed0 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:38.897123 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d506517c-e579-45ab-8cd2-b1f32f37fed0 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:38.897394 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d506517c-e579-45ab-8cd2-b1f32f37fed0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:38.898115 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d506517c-e579-45ab-8cd2-b1f32f37fed0 None admin] RBAC: Adding query filter params (group.id=f21c86cb093e4771896943da49969864, role.id=74a40bc819cb42eea7e966a4b2a016f1, scope.project.id=a4fe339e17324d15a22e51626818eecd) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:38.898685 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d506517c-e579-45ab-8cd2-b1f32f37fed0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:38.899841 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d506517c-e579-45ab-8cd2-b1f32f37fed0 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:38.906166 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 217/427] 10.222.0.61 () {56 vars in 1376 bytes} [Mon Aug 22 19:05:38 2022] GET /identity/v3/role_assignments?group.id=f21c86cb093e4771896943da49969864&scope.project.id=a4fe339e17324d15a22e51626818eecd&role.id=74a40bc819cb42eea7e966a4b2a016f1 => generated 611 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:38.908588 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ad23ba-7daa-4227-a757-ba884b5ac9d2 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:38.977436 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:39.036956 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ad23ba-7daa-4227-a757-ba884b5ac9d2 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:39.084667 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1528337c-fd2b-419d-80a1-81d1a543e50d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:39.085313 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1528337c-fd2b-419d-80a1-81d1a543e50d None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:39.085588 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1528337c-fd2b-419d-80a1-81d1a543e50d None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:39.085855 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1528337c-fd2b-419d-80a1-81d1a543e50d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:39.103235 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1528337c-fd2b-419d-80a1-81d1a543e50d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 19:05:39.113576 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-1528337c-fd2b-419d-80a1-81d1a543e50d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 22 19:05:39.113576 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:05:39.113576 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:05:39.113576 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:05:39.113576 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:05:39.113576 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:05:39.113576 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:05:39.113576 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:05:39.113576 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:05:39.113576 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:05:39.113576 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:05:39.113576 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:05:39.113576 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 19:05:39.113576 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 19:05:39.113576 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 19:05:39.113576 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 19:05:39.114719 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 19:05:39.114719 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 19:05:39.114719 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 19:05:39.114719 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 19:05:39.114719 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:05:39.114719 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:05:39.114719 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:05:39.114719 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:05:39.114719 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:05:39.114719 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:05:39.114719 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:05:39.114719 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:05:39.114719 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:05:39.114719 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:05:39.114719 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:05:39.114719 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:05:39.115965 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:05:39.115965 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:05:39.115965 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:05:39.115965 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:05:39.115965 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 19:05:39.115965 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 19:05:39.115965 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 19:05:39.115965 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 19:05:39.115965 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 19:05:39.115965 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 19:05:39.115965 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 22 19:05:39.115965 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:05:39.115965 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 211/428] 10.222.0.61 () {56 vars in 1099 bytes} [Mon Aug 22 19:05:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:05:39.122878 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d506517c-e579-45ab-8cd2-b1f32f37fed0 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:39.151217 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:39.167574 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1528337c-fd2b-419d-80a1-81d1a543e50d None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:39.298565 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d506517c-e579-45ab-8cd2-b1f32f37fed0 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:39.309222 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1528337c-fd2b-419d-80a1-81d1a543e50d None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:39.321010 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:39.347754 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ff407d-1bcb-4af1-8811-2451fa77b03c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:39.348734 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ff407d-1bcb-4af1-8811-2451fa77b03c None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:39.349248 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ff407d-1bcb-4af1-8811-2451fa77b03c None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:39.349770 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ff407d-1bcb-4af1-8811-2451fa77b03c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:39.350136 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944cc101-6799-4bf0-9d14-d72f337fc601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:39.350765 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944cc101-6799-4bf0-9d14-d72f337fc601 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:39.351041 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944cc101-6799-4bf0-9d14-d72f337fc601 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:39.351320 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944cc101-6799-4bf0-9d14-d72f337fc601 None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:39.352053 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944cc101-6799-4bf0-9d14-d72f337fc601 None admin] RBAC: Adding query filter params (name=member) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:39.352355 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944cc101-6799-4bf0-9d14-d72f337fc601 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:39.353765 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944cc101-6799-4bf0-9d14-d72f337fc601 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:39.366133 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ff407d-1bcb-4af1-8811-2451fa77b03c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 19:05:39.367798 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 218/429] 10.222.0.61 () {56 vars in 1101 bytes} [Mon Aug 22 19:05:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:39.371046 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944cc101-6799-4bf0-9d14-d72f337fc601 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:39.373128 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-60ff407d-1bcb-4af1-8811-2451fa77b03c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 19:05:39.373128 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:05:39.373128 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:05:39.373128 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:05:39.373128 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:05:39.373128 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:05:39.373128 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:05:39.373128 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:05:39.373128 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:05:39.373128 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:05:39.373128 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:05:39.373128 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:05:39.373128 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 19:05:39.373128 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 19:05:39.373128 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 19:05:39.373128 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 19:05:39.374043 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 19:05:39.374043 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 19:05:39.374043 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 19:05:39.374043 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 19:05:39.374043 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:05:39.374043 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:05:39.374043 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:05:39.374043 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:05:39.374043 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:05:39.374043 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:05:39.374043 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:05:39.374043 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:05:39.374043 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:05:39.374043 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:05:39.374043 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:05:39.374043 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:05:39.374894 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:05:39.374894 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:05:39.374894 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:05:39.374894 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:05:39.374894 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 19:05:39.374894 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 19:05:39.374894 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 19:05:39.374894 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 19:05:39.374894 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 19:05:39.374894 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 19:05:39.374894 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 19:05:39.374894 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:05:39.377482 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 212/430] 10.222.0.61 () {56 vars in 1114 bytes} [Mon Aug 22 19:05:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 19:05:39.386353 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ff407d-1bcb-4af1-8811-2451fa77b03c None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:39.484012 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:39.487699 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944cc101-6799-4bf0-9d14-d72f337fc601 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:39.518020 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ff407d-1bcb-4af1-8811-2451fa77b03c None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:39.527636 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4d3c84-9523-4593-affa-93ff329691e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:39.528103 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4d3c84-9523-4593-affa-93ff329691e5 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:39.528455 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4d3c84-9523-4593-affa-93ff329691e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:39.528844 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4d3c84-9523-4593-affa-93ff329691e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:39.562170 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c96f4c-9ee0-4274-8244-ef7a80189f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:39.563248 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c96f4c-9ee0-4274-8244-ef7a80189f67 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:39.564028 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c96f4c-9ee0-4274-8244-ef7a80189f67 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:39.564757 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c96f4c-9ee0-4274-8244-ef7a80189f67 None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:39.565609 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c96f4c-9ee0-4274-8244-ef7a80189f67 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:39.566103 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c96f4c-9ee0-4274-8244-ef7a80189f67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:39.568080 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c96f4c-9ee0-4274-8244-ef7a80189f67 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:39.579337 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 213/431] 10.222.0.61 () {56 vars in 1111 bytes} [Mon Aug 22 19:05:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:39.585182 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c96f4c-9ee0-4274-8244-ef7a80189f67 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:39.623759 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4d3c84-9523-4593-affa-93ff329691e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:39.625591 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4d3c84-9523-4593-affa-93ff329691e5 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:39.631003 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:39.719718 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:39.729931 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c96f4c-9ee0-4274-8244-ef7a80189f67 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:39.758630 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 219/432] 10.222.0.61 () {58 vars in 1282 bytes} [Mon Aug 22 19:05:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 22 19:05:39.760482 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4d3c84-9523-4593-affa-93ff329691e5 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:39.784547 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cda13df-83e9-432d-bccd-116721433b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:39.785420 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cda13df-83e9-432d-bccd-116721433b68 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:39.786346 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cda13df-83e9-432d-bccd-116721433b68 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:39.786776 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cda13df-83e9-432d-bccd-116721433b68 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:39.876105 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cda13df-83e9-432d-bccd-116721433b68 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:39.877486 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cda13df-83e9-432d-bccd-116721433b68 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:39.886280 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4d3c84-9523-4593-affa-93ff329691e5 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:39.912430 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:39.930256 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15fb629-34f9-4585-be34-aa802b7cfd83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:39.930918 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15fb629-34f9-4585-be34-aa802b7cfd83 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:39.931172 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15fb629-34f9-4585-be34-aa802b7cfd83 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:39.931442 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15fb629-34f9-4585-be34-aa802b7cfd83 None admin] PATH_INFO: `/v3/projects/47adea686aa0431fb9766ac68f4bf165` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:39.932676 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15fb629-34f9-4585-be34-aa802b7cfd83 None admin] RBAC: Authorizing `identity:get_project(project_id=47adea686aa0431fb9766ac68f4bf165)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:39.934128 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15fb629-34f9-4585-be34-aa802b7cfd83 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:39.939762 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 220/433] 10.222.0.61 () {56 vars in 1186 bytes} [Mon Aug 22 19:05:39 2022] GET /identity/v3/projects/47adea686aa0431fb9766ac68f4bf165 => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:39.941835 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15fb629-34f9-4585-be34-aa802b7cfd83 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:39.994270 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:39.994614 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 214/434] 10.222.0.61 () {58 vars in 1282 bytes} [Mon Aug 22 19:05:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 22 19:05:40.001197 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cda13df-83e9-432d-bccd-116721433b68 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:40.063664 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15fb629-34f9-4585-be34-aa802b7cfd83 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:40.100065 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a7c22f-42cf-4e75-8efd-356eb92196b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:40.100705 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a7c22f-42cf-4e75-8efd-356eb92196b2 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:40.100986 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a7c22f-42cf-4e75-8efd-356eb92196b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:40.101260 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a7c22f-42cf-4e75-8efd-356eb92196b2 None admin] PATH_INFO: `/v3/groups/f21c86cb093e4771896943da49969864` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:40.102405 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a7c22f-42cf-4e75-8efd-356eb92196b2 None admin] RBAC: Authorizing `identity:get_group(group_id=f21c86cb093e4771896943da49969864)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:40.103970 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a7c22f-42cf-4e75-8efd-356eb92196b2 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:40.105836 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 221/435] 10.222.0.61 () {56 vars in 1180 bytes} [Mon Aug 22 19:05:39 2022] GET /identity/v3/groups/f21c86cb093e4771896943da49969864 => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 19:05:40.113234 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a7c22f-42cf-4e75-8efd-356eb92196b2 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:40.130620 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cda13df-83e9-432d-bccd-116721433b68 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:40.163769 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:40.170946 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00250a4-2287-42d4-a891-9d81a203de3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:40.171908 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00250a4-2287-42d4-a891-9d81a203de3b None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:40.172186 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00250a4-2287-42d4-a891-9d81a203de3b None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:40.172462 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00250a4-2287-42d4-a891-9d81a203de3b None admin] PATH_INFO: `/v3/projects/a4fe339e17324d15a22e51626818eecd` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:40.173869 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00250a4-2287-42d4-a891-9d81a203de3b None admin] RBAC: Authorizing `identity:get_project(project_id=a4fe339e17324d15a22e51626818eecd)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:40.176130 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00250a4-2287-42d4-a891-9d81a203de3b None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:40.178767 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 215/436] 10.222.0.61 () {56 vars in 1186 bytes} [Mon Aug 22 19:05:39 2022] GET /identity/v3/projects/a4fe339e17324d15a22e51626818eecd => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:40.185165 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00250a4-2287-42d4-a891-9d81a203de3b None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:40.236005 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a7c22f-42cf-4e75-8efd-356eb92196b2 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:40.247143 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:40.273431 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03fadc6-2600-45c7-b15b-f812e9b9c61e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:40.274044 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03fadc6-2600-45c7-b15b-f812e9b9c61e None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:40.274328 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03fadc6-2600-45c7-b15b-f812e9b9c61e None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:40.274645 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03fadc6-2600-45c7-b15b-f812e9b9c61e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:40.275319 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03fadc6-2600-45c7-b15b-f812e9b9c61e None admin] RBAC: Adding query filter params (group.id=f21c86cb093e4771896943da49969864, role.id=74a40bc819cb42eea7e966a4b2a016f1, scope.project.id=47adea686aa0431fb9766ac68f4bf165) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:40.275622 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03fadc6-2600-45c7-b15b-f812e9b9c61e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:40.277129 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03fadc6-2600-45c7-b15b-f812e9b9c61e None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:40.290259 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 222/437] 10.222.0.61 () {56 vars in 1376 bytes} [Mon Aug 22 19:05:40 2022] GET /identity/v3/role_assignments?group.id=f21c86cb093e4771896943da49969864&scope.project.id=47adea686aa0431fb9766ac68f4bf165&role.id=74a40bc819cb42eea7e966a4b2a016f1 => generated 611 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:40.290259 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095debe9-1b6a-4bde-bb79-d995a0c58adb None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:40.290259 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095debe9-1b6a-4bde-bb79-d995a0c58adb None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:40.290259 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095debe9-1b6a-4bde-bb79-d995a0c58adb None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:40.290259 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 223/438] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:05:40 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:05:40.291977 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9351de-b79a-43c0-b7a2-bc8e0b5e3879 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:40.292485 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9351de-b79a-43c0-b7a2-bc8e0b5e3879 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:40.292598 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9351de-b79a-43c0-b7a2-bc8e0b5e3879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:40.302332 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00250a4-2287-42d4-a891-9d81a203de3b None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:40.314176 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-5f9351de-b79a-43c0-b7a2-bc8e0b5e3879 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:05:40.315866 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-5f9351de-b79a-43c0-b7a2-bc8e0b5e3879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:05:40.341178 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9579c7da-d595-4efe-8a1b-f9220a2b2735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:40.341833 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9579c7da-d595-4efe-8a1b-f9220a2b2735 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:40.342111 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9579c7da-d595-4efe-8a1b-f9220a2b2735 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:40.342378 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9579c7da-d595-4efe-8a1b-f9220a2b2735 None admin] PATH_INFO: `/v3/groups/f21c86cb093e4771896943da49969864` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:40.344266 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9579c7da-d595-4efe-8a1b-f9220a2b2735 None admin] RBAC: Authorizing `identity:get_group(group_id=f21c86cb093e4771896943da49969864)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:40.345762 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9579c7da-d595-4efe-8a1b-f9220a2b2735 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:40.347443 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 216/439] 10.222.0.61 () {56 vars in 1180 bytes} [Mon Aug 22 19:05:40 2022] GET /identity/v3/groups/f21c86cb093e4771896943da49969864 => generated 226 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:40.348656 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e53525-97f0-40e4-8050-b5a0dcf6a424 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:40.348935 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e53525-97f0-40e4-8050-b5a0dcf6a424 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:40.349206 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e53525-97f0-40e4-8050-b5a0dcf6a424 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:40.350421 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 217/440] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:05:40 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:05:40.357202 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e53525-97f0-40e4-8050-b5a0dcf6a424 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:40.411902 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 224/441] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:05:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 19:05:40.413266 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cc3a4e-ec0d-4436-a694-fdf9366af85a None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:40.413630 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cc3a4e-ec0d-4436-a694-fdf9366af85a None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:40.413975 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cc3a4e-ec0d-4436-a694-fdf9366af85a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:40.434939 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-c8cc3a4e-ec0d-4436-a694-fdf9366af85a None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:05:40.436405 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-c8cc3a4e-ec0d-4436-a694-fdf9366af85a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:05:40.441732 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:40.491436 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e53525-97f0-40e4-8050-b5a0dcf6a424 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:40.534868 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 225/442] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:05:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 19:05:40.536187 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66693c1-5283-439e-9e64-5cddc4c27553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:40.536872 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66693c1-5283-439e-9e64-5cddc4c27553 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:40.537220 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66693c1-5283-439e-9e64-5cddc4c27553 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:40.537559 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66693c1-5283-439e-9e64-5cddc4c27553 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:40.538387 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66693c1-5283-439e-9e64-5cddc4c27553 None admin] RBAC: Adding query filter params (group.id=f21c86cb093e4771896943da49969864, role.id=9f0952608a704dfca5fdbe4046331f21, scope.project.id=a4fe339e17324d15a22e51626818eecd) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:40.538774 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66693c1-5283-439e-9e64-5cddc4c27553 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:40.540305 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66693c1-5283-439e-9e64-5cddc4c27553 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:40.547707 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 218/443] 10.222.0.61 () {56 vars in 1376 bytes} [Mon Aug 22 19:05:40 2022] GET /identity/v3/role_assignments?group.id=f21c86cb093e4771896943da49969864&scope.project.id=a4fe339e17324d15a22e51626818eecd&role.id=9f0952608a704dfca5fdbe4046331f21 => generated 611 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:40.688049 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cc3a4e-ec0d-4436-a694-fdf9366af85a None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:40.724503 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:40.797700 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cc3a4e-ec0d-4436-a694-fdf9366af85a None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:40.804107 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66693c1-5283-439e-9e64-5cddc4c27553 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:40.840406 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b97d4a-45f3-4acc-9fbf-133f772cb710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:40.841496 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b97d4a-45f3-4acc-9fbf-133f772cb710 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:40.841783 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b97d4a-45f3-4acc-9fbf-133f772cb710 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:40.842360 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b97d4a-45f3-4acc-9fbf-133f772cb710 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:40.853300 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b97d4a-45f3-4acc-9fbf-133f772cb710 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 19:05:40.858139 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-c4b97d4a-45f3-4acc-9fbf-133f772cb710 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 19:05:40.858139 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:05:40.858139 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:05:40.858139 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:05:40.858139 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:05:40.858139 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:05:40.858139 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:05:40.858139 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:05:40.858139 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:05:40.858139 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:05:40.858139 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:05:40.858139 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:05:40.858139 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 19:05:40.858139 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 19:05:40.858139 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 19:05:40.858139 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 19:05:40.859065 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 19:05:40.859065 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 19:05:40.859065 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 19:05:40.859065 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 19:05:40.859065 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:05:40.859065 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:05:40.859065 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:05:40.859065 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:05:40.859065 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:05:40.859065 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:05:40.859065 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:05:40.859065 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:05:40.859065 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:05:40.859065 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:05:40.859065 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:05:40.859065 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:05:40.859950 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:05:40.859950 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:05:40.859950 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:05:40.859950 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:05:40.859950 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 19:05:40.859950 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 19:05:40.859950 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 19:05:40.859950 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 19:05:40.859950 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 19:05:40.859950 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 19:05:40.859950 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 19:05:40.859950 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:05:40.859950 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 226/444] 10.222.0.61 () {56 vars in 1114 bytes} [Mon Aug 22 19:05:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:05:40.865255 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b97d4a-45f3-4acc-9fbf-133f772cb710 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:40.932156 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66693c1-5283-439e-9e64-5cddc4c27553 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:40.969110 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:40.975098 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1b8ca7-cedd-4d6a-a332-b3ad7123d170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:40.975783 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1b8ca7-cedd-4d6a-a332-b3ad7123d170 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:40.976066 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1b8ca7-cedd-4d6a-a332-b3ad7123d170 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:40.976400 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1b8ca7-cedd-4d6a-a332-b3ad7123d170 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:40.981434 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:40.991409 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1b8ca7-cedd-4d6a-a332-b3ad7123d170 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 19:05:40.991826 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b97d4a-45f3-4acc-9fbf-133f772cb710 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:40.996713 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-4f1b8ca7-cedd-4d6a-a332-b3ad7123d170 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 19:05:40.996713 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:05:40.996713 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:05:40.996713 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:05:40.996713 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:05:40.996713 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:05:40.996713 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:05:40.996713 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:05:40.996713 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:05:40.996713 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:05:40.996713 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:05:40.996713 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:05:40.996713 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 19:05:40.996713 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 19:05:40.996713 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 19:05:40.996713 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 19:05:40.997639 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 19:05:40.997639 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 19:05:40.997639 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 19:05:40.997639 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 19:05:40.997639 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:05:40.997639 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:05:40.997639 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:05:40.997639 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:05:40.997639 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:05:40.997639 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:05:40.997639 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:05:40.997639 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:05:40.997639 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:05:40.997639 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:05:40.997639 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:05:40.997639 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:05:40.998517 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:05:40.998517 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:05:40.998517 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:05:40.998517 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:05:40.998517 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 19:05:40.998517 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 19:05:40.998517 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 19:05:40.998517 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 19:05:40.998517 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 19:05:40.998517 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 19:05:40.998517 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 19:05:40.998517 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:05:40.998517 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 219/445] 10.222.0.61 () {56 vars in 1096 bytes} [Mon Aug 22 19:05:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 19:05:41.005133 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1b8ca7-cedd-4d6a-a332-b3ad7123d170 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:41.034548 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50966361-4977-40ff-8360-e12535a61365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:41.035164 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50966361-4977-40ff-8360-e12535a61365 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:41.035446 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50966361-4977-40ff-8360-e12535a61365 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:41.035757 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50966361-4977-40ff-8360-e12535a61365 None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:41.036573 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50966361-4977-40ff-8360-e12535a61365 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:41.036731 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50966361-4977-40ff-8360-e12535a61365 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:41.038112 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50966361-4977-40ff-8360-e12535a61365 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:41.047436 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 227/446] 10.222.0.61 () {56 vars in 1111 bytes} [Mon Aug 22 19:05:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 19:05:41.054186 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50966361-4977-40ff-8360-e12535a61365 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:41.112249 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1b8ca7-cedd-4d6a-a332-b3ad7123d170 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:41.147444 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b71245-93e8-4738-9517-9bdd75571135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:41.148085 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b71245-93e8-4738-9517-9bdd75571135 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:41.148499 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b71245-93e8-4738-9517-9bdd75571135 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:41.148636 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b71245-93e8-4738-9517-9bdd75571135 None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:41.149304 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b71245-93e8-4738-9517-9bdd75571135 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:41.149606 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b71245-93e8-4738-9517-9bdd75571135 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:41.150964 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b71245-93e8-4738-9517-9bdd75571135 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:41.157643 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50966361-4977-40ff-8360-e12535a61365 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:41.159396 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 220/447] 10.222.0.61 () {56 vars in 1099 bytes} [Mon Aug 22 19:05:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 19:05:41.164816 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b71245-93e8-4738-9517-9bdd75571135 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:41.191713 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1039ac7c-6b60-43e8-b791-d20c3e7f9b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:41.192325 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1039ac7c-6b60-43e8-b791-d20c3e7f9b7d None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:41.192604 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1039ac7c-6b60-43e8-b791-d20c3e7f9b7d None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:41.192878 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1039ac7c-6b60-43e8-b791-d20c3e7f9b7d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:41.218437 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:41.220765 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:41.262821 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1039ac7c-6b60-43e8-b791-d20c3e7f9b7d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:41.264296 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1039ac7c-6b60-43e8-b791-d20c3e7f9b7d None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:41.266717 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b71245-93e8-4738-9517-9bdd75571135 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:41.299087 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a785cb-6c93-4fcd-a444-757f59fab524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:41.299692 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a785cb-6c93-4fcd-a444-757f59fab524 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:41.299973 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a785cb-6c93-4fcd-a444-757f59fab524 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:41.300241 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a785cb-6c93-4fcd-a444-757f59fab524 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:41.362210 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 228/448] 10.222.0.61 () {58 vars in 1282 bytes} [Mon Aug 22 19:05:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 19:05:41.366074 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a785cb-6c93-4fcd-a444-757f59fab524 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:41.367629 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a785cb-6c93-4fcd-a444-757f59fab524 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:41.368264 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1039ac7c-6b60-43e8-b791-d20c3e7f9b7d None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:41.420190 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:41.423171 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:41.465110 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1039ac7c-6b60-43e8-b791-d20c3e7f9b7d None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:41.465248 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 221/449] 10.222.0.61 () {58 vars in 1282 bytes} [Mon Aug 22 19:05:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 22 19:05:41.471604 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a785cb-6c93-4fcd-a444-757f59fab524 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:41.500217 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d18a7c-8182-4c49-88cd-a7bd44343ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:41.500853 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d18a7c-8182-4c49-88cd-a7bd44343ee4 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:41.501132 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d18a7c-8182-4c49-88cd-a7bd44343ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:41.501403 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d18a7c-8182-4c49-88cd-a7bd44343ee4 None admin] PATH_INFO: `/v3/projects/47adea686aa0431fb9766ac68f4bf165` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:41.502552 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d18a7c-8182-4c49-88cd-a7bd44343ee4 None admin] RBAC: Authorizing `identity:get_project(project_id=47adea686aa0431fb9766ac68f4bf165)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:41.504030 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d18a7c-8182-4c49-88cd-a7bd44343ee4 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:41.505654 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 229/450] 10.222.0.61 () {56 vars in 1186 bytes} [Mon Aug 22 19:05:41 2022] GET /identity/v3/projects/47adea686aa0431fb9766ac68f4bf165 => generated 302 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 19:05:41.510999 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d18a7c-8182-4c49-88cd-a7bd44343ee4 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:41.568908 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a785cb-6c93-4fcd-a444-757f59fab524 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:41.602117 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332c9170-ec1f-405d-bda7-b62a5688887c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:41.602737 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332c9170-ec1f-405d-bda7-b62a5688887c None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:41.603014 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332c9170-ec1f-405d-bda7-b62a5688887c None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:41.603283 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332c9170-ec1f-405d-bda7-b62a5688887c None admin] PATH_INFO: `/v3/projects/4bc536d83f9841a59997d0d04e993cd8` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:41.604467 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332c9170-ec1f-405d-bda7-b62a5688887c None admin] RBAC: Authorizing `identity:get_project(project_id=4bc536d83f9841a59997d0d04e993cd8)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:41.605880 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332c9170-ec1f-405d-bda7-b62a5688887c None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:41.607499 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 222/451] 10.222.0.61 () {56 vars in 1186 bytes} [Mon Aug 22 19:05:41 2022] GET /identity/v3/projects/4bc536d83f9841a59997d0d04e993cd8 => generated 344 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 19:05:41.610435 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d18a7c-8182-4c49-88cd-a7bd44343ee4 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:41.613443 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332c9170-ec1f-405d-bda7-b62a5688887c None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:41.634415 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:41.637313 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:41.645457 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7381ee5-0ad2-4567-8ba2-3f51d2c53d01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:41.646080 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7381ee5-0ad2-4567-8ba2-3f51d2c53d01 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:41.646358 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7381ee5-0ad2-4567-8ba2-3f51d2c53d01 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:41.646629 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7381ee5-0ad2-4567-8ba2-3f51d2c53d01 None admin] PATH_INFO: `/v3/groups/f21c86cb093e4771896943da49969864` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:41.647774 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7381ee5-0ad2-4567-8ba2-3f51d2c53d01 None admin] RBAC: Authorizing `identity:get_group(group_id=f21c86cb093e4771896943da49969864)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:41.649199 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7381ee5-0ad2-4567-8ba2-3f51d2c53d01 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:41.650751 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 230/452] 10.222.0.61 () {56 vars in 1180 bytes} [Mon Aug 22 19:05:41 2022] GET /identity/v3/groups/f21c86cb093e4771896943da49969864 => generated 226 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 19:05:41.657011 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7381ee5-0ad2-4567-8ba2-3f51d2c53d01 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:41.713554 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332c9170-ec1f-405d-bda7-b62a5688887c None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:41.747540 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bfc673-5319-423e-8452-5287bd4c0977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:41.748136 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bfc673-5319-423e-8452-5287bd4c0977 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:41.748412 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bfc673-5319-423e-8452-5287bd4c0977 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:41.748681 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bfc673-5319-423e-8452-5287bd4c0977 None admin] PATH_INFO: `/v3/groups/111ff86c43de4c45a0be30d3e1a0ea46` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:41.749817 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bfc673-5319-423e-8452-5287bd4c0977 None admin] RBAC: Authorizing `identity:get_group(group_id=111ff86c43de4c45a0be30d3e1a0ea46)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:41.751264 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bfc673-5319-423e-8452-5287bd4c0977 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:41.752939 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 223/453] 10.222.0.61 () {56 vars in 1180 bytes} [Mon Aug 22 19:05:41 2022] GET /identity/v3/groups/111ff86c43de4c45a0be30d3e1a0ea46 => generated 229 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 19:05:41.755164 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7381ee5-0ad2-4567-8ba2-3f51d2c53d01 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:41.758541 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bfc673-5319-423e-8452-5287bd4c0977 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:41.789051 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed8f28a-4a11-4b40-98bb-82512fb70bd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:41.789652 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed8f28a-4a11-4b40-98bb-82512fb70bd1 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:41.789928 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed8f28a-4a11-4b40-98bb-82512fb70bd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:41.790201 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed8f28a-4a11-4b40-98bb-82512fb70bd1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:41.790923 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed8f28a-4a11-4b40-98bb-82512fb70bd1 None admin] RBAC: Adding query filter params (group.id=f21c86cb093e4771896943da49969864, role.id=9f0952608a704dfca5fdbe4046331f21, scope.project.id=47adea686aa0431fb9766ac68f4bf165) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:41.791227 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed8f28a-4a11-4b40-98bb-82512fb70bd1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:41.792672 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed8f28a-4a11-4b40-98bb-82512fb70bd1 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:41.811674 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 231/454] 10.222.0.61 () {56 vars in 1376 bytes} [Mon Aug 22 19:05:41 2022] GET /identity/v3/role_assignments?group.id=f21c86cb093e4771896943da49969864&scope.project.id=47adea686aa0431fb9766ac68f4bf165&role.id=9f0952608a704dfca5fdbe4046331f21 => generated 611 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 19:05:41.852410 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:41.865325 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bfc673-5319-423e-8452-5287bd4c0977 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:41.901848 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85447045-23de-48d4-bb24-af7e73cd9164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:41.902469 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85447045-23de-48d4-bb24-af7e73cd9164 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:41.902750 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85447045-23de-48d4-bb24-af7e73cd9164 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:41.903012 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85447045-23de-48d4-bb24-af7e73cd9164 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:41.903781 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85447045-23de-48d4-bb24-af7e73cd9164 None admin] RBAC: Adding query filter params (group.id=111ff86c43de4c45a0be30d3e1a0ea46, role.id=163c30c3b3c246beb849ce176f02d9da, scope.project.id=4bc536d83f9841a59997d0d04e993cd8) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:41.904089 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85447045-23de-48d4-bb24-af7e73cd9164 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:41.905536 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85447045-23de-48d4-bb24-af7e73cd9164 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:41.912633 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 224/455] 10.222.0.61 () {56 vars in 1376 bytes} [Mon Aug 22 19:05:41 2022] GET /identity/v3/role_assignments?group.id=111ff86c43de4c45a0be30d3e1a0ea46&scope.project.id=4bc536d83f9841a59997d0d04e993cd8&role.id=163c30c3b3c246beb849ce176f02d9da => generated 611 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 19:05:44.158957 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a535ea-a574-454b-a408-e09cdc5f61c1 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:44.159474 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a535ea-a574-454b-a408-e09cdc5f61c1 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:44.159474 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a535ea-a574-454b-a408-e09cdc5f61c1 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:44.161060 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 232/456] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:05:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:05:44.167822 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701e20e0-a24b-43d0-b3b3-911b9a943a37 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:44.168648 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701e20e0-a24b-43d0-b3b3-911b9a943a37 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:44.169122 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701e20e0-a24b-43d0-b3b3-911b9a943a37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:44.189568 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-701e20e0-a24b-43d0-b3b3-911b9a943a37 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:05:44.191109 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-701e20e0-a24b-43d0-b3b3-911b9a943a37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:05:44.275723 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 225/457] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:05:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 19:05:44.525556 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a0dacc-bd3d-488c-b6f0-e383772d4a07 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:44.526248 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a0dacc-bd3d-488c-b6f0-e383772d4a07 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:44.526758 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a0dacc-bd3d-488c-b6f0-e383772d4a07 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:44.528307 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 233/458] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:05:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:05:44.534161 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521c0b49-5287-4bbe-a051-9f0e2824f1bd None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:44.534676 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521c0b49-5287-4bbe-a051-9f0e2824f1bd None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:44.535151 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521c0b49-5287-4bbe-a051-9f0e2824f1bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:44.559765 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-521c0b49-5287-4bbe-a051-9f0e2824f1bd None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:05:44.561270 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-521c0b49-5287-4bbe-a051-9f0e2824f1bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:05:44.645908 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 226/459] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:05:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 19:05:44.737035 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a0dacc-bd3d-488c-b6f0-e383772d4a07 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:44.804459 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:44.868316 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a0dacc-bd3d-488c-b6f0-e383772d4a07 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:44.913918 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c16613-d282-4fdd-aeeb-91741409b89d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:44.914116 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84e51aa-d74a-4612-ad26-ad87843e9de9 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:44.914199 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84e51aa-d74a-4612-ad26-ad87843e9de9 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:44.914492 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84e51aa-d74a-4612-ad26-ad87843e9de9 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:44.915099 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c16613-d282-4fdd-aeeb-91741409b89d None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:44.915376 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c16613-d282-4fdd-aeeb-91741409b89d None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:44.915680 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c16613-d282-4fdd-aeeb-91741409b89d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:44.917989 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 227/460] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:05:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 19:05:44.923559 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7355c9b8-24a7-48dd-b798-b8f1db09d2ba None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:44.924234 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7355c9b8-24a7-48dd-b798-b8f1db09d2ba None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:44.924724 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7355c9b8-24a7-48dd-b798-b8f1db09d2ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:44.932715 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c16613-d282-4fdd-aeeb-91741409b89d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:44.935268 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c16613-d282-4fdd-aeeb-91741409b89d None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:44.946025 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-7355c9b8-24a7-48dd-b798-b8f1db09d2ba None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:05:44.947635 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-7355c9b8-24a7-48dd-b798-b8f1db09d2ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:05:44.956884 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-d4c16613-d282-4fdd-aeeb-91741409b89d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:05:44.956884 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:05:44.956884 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:05:44.956884 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:05:44.956884 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:05:44.956884 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:05:44.956884 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:05:44.956884 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:05:44.956884 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:05:44.956884 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:05:44.956884 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:05:44.956884 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:05:44.956884 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 19:05:44.956884 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 19:05:44.956884 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 19:05:44.956884 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 19:05:44.957808 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:05:44.957808 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:05:44.957808 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:05:44.957808 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:05:44.957808 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:05:44.957808 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:05:44.957808 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:05:44.957808 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:05:44.957808 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:05:44.957808 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:05:44.957808 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:05:44.957808 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:05:44.957808 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:05:44.957808 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:05:44.957808 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:05:44.957808 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:05:44.957808 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 19:05:44.959228 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 19:05:44.959228 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:05:44.959228 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:05:44.959228 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 234/461] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:05:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:05:44.959556 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86032a12-992f-4668-b840-e5994d04c30e None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:44.959830 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86032a12-992f-4668-b840-e5994d04c30e None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:44.960102 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86032a12-992f-4668-b840-e5994d04c30e None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:44.961840 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:44.962016 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 235/462] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:05:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:05:44.967446 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86032a12-992f-4668-b840-e5994d04c30e None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:45.025805 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 228/463] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:05:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 19:05:45.027908 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512bab5c-b452-4bbd-b23e-c3a00b9d16d7 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:45.028405 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512bab5c-b452-4bbd-b23e-c3a00b9d16d7 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:45.028891 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512bab5c-b452-4bbd-b23e-c3a00b9d16d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:45.054409 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-512bab5c-b452-4bbd-b23e-c3a00b9d16d7 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:05:45.056614 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-512bab5c-b452-4bbd-b23e-c3a00b9d16d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:05:45.078360 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86032a12-992f-4668-b840-e5994d04c30e None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:45.087758 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:45.118657 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f043e06-3eaf-4bbf-906e-ad97ed631d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:45.119313 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f043e06-3eaf-4bbf-906e-ad97ed631d45 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:45.119596 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f043e06-3eaf-4bbf-906e-ad97ed631d45 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:45.119954 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f043e06-3eaf-4bbf-906e-ad97ed631d45 None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:45.120599 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f043e06-3eaf-4bbf-906e-ad97ed631d45 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:45.120943 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f043e06-3eaf-4bbf-906e-ad97ed631d45 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:45.122386 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f043e06-3eaf-4bbf-906e-ad97ed631d45 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:45.135581 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 236/464] 10.222.0.61 () {56 vars in 1109 bytes} [Mon Aug 22 19:05:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:45.136480 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f043e06-3eaf-4bbf-906e-ad97ed631d45 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:45.161663 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 229/465] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:05:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 19:05:45.163213 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512bab5c-b452-4bbd-b23e-c3a00b9d16d7 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:45.268694 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:45.304945 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512bab5c-b452-4bbd-b23e-c3a00b9d16d7 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:45.340705 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f043e06-3eaf-4bbf-906e-ad97ed631d45 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:45.354627 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cfa600-0f2b-4b2a-ab0b-a235528de57f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:45.355246 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cfa600-0f2b-4b2a-ab0b-a235528de57f None admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:45.355525 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cfa600-0f2b-4b2a-ab0b-a235528de57f None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:45.355834 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cfa600-0f2b-4b2a-ab0b-a235528de57f None admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:45.356487 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cfa600-0f2b-4b2a-ab0b-a235528de57f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:45.358077 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cfa600-0f2b-4b2a-ab0b-a235528de57f None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:45.381365 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3445df-3590-43ce-8ebf-29ee78c3196d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:45.382331 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3445df-3590-43ce-8ebf-29ee78c3196d None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:45.382814 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3445df-3590-43ce-8ebf-29ee78c3196d None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:45.383313 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3445df-3590-43ce-8ebf-29ee78c3196d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:45.389008 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 230/466] 10.222.0.61 () {60 vars in 1132 bytes} [Mon Aug 22 19:05:45 2022] POST /identity/v3/users => generated 246 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:05:45.390344 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cfa600-0f2b-4b2a-ab0b-a235528de57f None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:45.408634 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3445df-3590-43ce-8ebf-29ee78c3196d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:45.410616 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3445df-3590-43ce-8ebf-29ee78c3196d None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:45.444000 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-be3445df-3590-43ce-8ebf-29ee78c3196d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:05:45.444000 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:05:45.444000 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:05:45.444000 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:05:45.444000 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:05:45.444000 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:05:45.444000 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:05:45.444000 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:05:45.444000 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:05:45.444000 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:05:45.444000 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:05:45.444000 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:05:45.444000 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 19:05:45.444000 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 19:05:45.444000 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 19:05:45.444000 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 19:05:45.444997 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:05:45.444997 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:05:45.444997 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:05:45.444997 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:05:45.444997 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:05:45.444997 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:05:45.444997 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:05:45.444997 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:05:45.444997 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:05:45.444997 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:05:45.444997 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:05:45.444997 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:05:45.444997 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:05:45.444997 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:05:45.444997 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:05:45.444997 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:05:45.444997 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 19:05:45.445860 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 19:05:45.445860 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:05:45.445860 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:05:45.450691 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 237/467] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:05:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:05:45.455062 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3445df-3590-43ce-8ebf-29ee78c3196d None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:45.486508 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:45.556414 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cfa600-0f2b-4b2a-ab0b-a235528de57f None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:45.583460 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3445df-3590-43ce-8ebf-29ee78c3196d None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:45.591497 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:45.601645 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cb712e-7b3f-4089-a3e8-56774ccb76e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:45.602281 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb712e-7b3f-4089-a3e8-56774ccb76e5 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:45.602793 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb712e-7b3f-4089-a3e8-56774ccb76e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:45.603377 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb712e-7b3f-4089-a3e8-56774ccb76e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:45.618935 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cb712e-7b3f-4089-a3e8-56774ccb76e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:45.621033 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cb712e-7b3f-4089-a3e8-56774ccb76e5 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:45.629743 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e052f6a8-0583-42cc-ac70-a5d1cdf9fc65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:45.630672 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e052f6a8-0583-42cc-ac70-a5d1cdf9fc65 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:45.631155 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e052f6a8-0583-42cc-ac70-a5d1cdf9fc65 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:45.631492 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e052f6a8-0583-42cc-ac70-a5d1cdf9fc65 None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:45.632257 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e052f6a8-0583-42cc-ac70-a5d1cdf9fc65 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:45.632561 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e052f6a8-0583-42cc-ac70-a5d1cdf9fc65 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:45.633950 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e052f6a8-0583-42cc-ac70-a5d1cdf9fc65 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:45.638165 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-d8cb712e-7b3f-4089-a3e8-56774ccb76e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:05:45.638165 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:05:45.638165 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:05:45.638165 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:05:45.638165 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:05:45.638165 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:05:45.638165 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:05:45.638165 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:05:45.638165 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:05:45.638165 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:05:45.638165 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:05:45.638165 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:05:45.638165 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 19:05:45.638165 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 19:05:45.638165 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 19:05:45.638165 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 19:05:45.639071 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:05:45.639071 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:05:45.639071 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:05:45.639071 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:05:45.639071 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:05:45.639071 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:05:45.639071 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:05:45.639071 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:05:45.639071 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:05:45.639071 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:05:45.639071 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:05:45.639071 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:05:45.639071 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:05:45.639071 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:05:45.639071 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:05:45.639071 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:05:45.639071 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 19:05:45.641105 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 19:05:45.641105 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:05:45.641105 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:05:45.641105 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 231/468] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:05:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:05:45.646334 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cb712e-7b3f-4089-a3e8-56774ccb76e5 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:45.650743 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 238/469] 10.222.0.61 () {56 vars in 1109 bytes} [Mon Aug 22 19:05:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:45.651419 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e052f6a8-0583-42cc-ac70-a5d1cdf9fc65 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:45.745632 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:45.765969 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e052f6a8-0583-42cc-ac70-a5d1cdf9fc65 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:45.774918 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cb712e-7b3f-4089-a3e8-56774ccb76e5 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:45.802105 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1805d68b-c4bb-4fb0-952b-bc25a8a0162b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:45.802720 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1805d68b-c4bb-4fb0-952b-bc25a8a0162b None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:45.803005 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1805d68b-c4bb-4fb0-952b-bc25a8a0162b None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:45.803279 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1805d68b-c4bb-4fb0-952b-bc25a8a0162b None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:45.803955 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1805d68b-c4bb-4fb0-952b-bc25a8a0162b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:45.804269 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1805d68b-c4bb-4fb0-952b-bc25a8a0162b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:45.805619 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1805d68b-c4bb-4fb0-952b-bc25a8a0162b None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:45.811247 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a86c5a4-87e5-438d-aabc-77a2ade238f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:45.811893 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a86c5a4-87e5-438d-aabc-77a2ade238f4 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:45.812491 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a86c5a4-87e5-438d-aabc-77a2ade238f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:45.812491 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a86c5a4-87e5-438d-aabc-77a2ade238f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:45.817449 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 239/470] 10.222.0.61 () {56 vars in 1109 bytes} [Mon Aug 22 19:05:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:45.822794 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1805d68b-c4bb-4fb0-952b-bc25a8a0162b None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:45.823787 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a86c5a4-87e5-438d-aabc-77a2ade238f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:45.825235 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a86c5a4-87e5-438d-aabc-77a2ade238f4 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:45.840665 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-4a86c5a4-87e5-438d-aabc-77a2ade238f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:05:45.840665 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:05:45.840665 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:05:45.840665 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:05:45.840665 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:05:45.840665 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:05:45.840665 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:05:45.840665 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:05:45.840665 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:05:45.840665 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:05:45.840665 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:05:45.840665 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:05:45.840665 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 19:05:45.840665 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 19:05:45.840665 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 19:05:45.840665 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 19:05:45.841642 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:05:45.841642 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:05:45.841642 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:05:45.841642 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:05:45.841642 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:05:45.841642 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:05:45.841642 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:05:45.841642 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:05:45.841642 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:05:45.841642 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:05:45.841642 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:05:45.841642 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:05:45.841642 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:05:45.841642 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:05:45.841642 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:05:45.841642 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:05:45.841642 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 19:05:45.842525 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 19:05:45.842525 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:05:45.842525 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:05:45.842525 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 232/471] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:05:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:05:45.843297 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf808d5-ea0a-442d-9a74-2b300ee590d6 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:45.843297 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf808d5-ea0a-442d-9a74-2b300ee590d6 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:45.843604 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf808d5-ea0a-442d-9a74-2b300ee590d6 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:45.847690 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 233/472] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:05:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:05:45.847690 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf808d5-ea0a-442d-9a74-2b300ee590d6 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:45.885643 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:45.940061 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1805d68b-c4bb-4fb0-952b-bc25a8a0162b None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:45.965744 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf808d5-ea0a-442d-9a74-2b300ee590d6 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:45.976086 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85078d59-abdc-4d9e-830b-a391549145f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:45.976704 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85078d59-abdc-4d9e-830b-a391549145f2 None admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:45.977109 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85078d59-abdc-4d9e-830b-a391549145f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:45.977382 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85078d59-abdc-4d9e-830b-a391549145f2 None admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:45.978025 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85078d59-abdc-4d9e-830b-a391549145f2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:45.979688 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85078d59-abdc-4d9e-830b-a391549145f2 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:45.992383 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:46.004017 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c517b92-0cf3-4746-833e-e4f9855054d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:46.004631 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c517b92-0cf3-4746-833e-e4f9855054d7 None admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:46.004907 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c517b92-0cf3-4746-833e-e4f9855054d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:46.005178 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c517b92-0cf3-4746-833e-e4f9855054d7 None admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:46.005878 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c517b92-0cf3-4746-833e-e4f9855054d7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:46.007350 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c517b92-0cf3-4746-833e-e4f9855054d7 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:46.016446 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 240/473] 10.222.0.61 () {60 vars in 1132 bytes} [Mon Aug 22 19:05:45 2022] POST /identity/v3/users => generated 248 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:05:46.017245 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85078d59-abdc-4d9e-830b-a391549145f2 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:46.038424 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 234/474] 10.222.0.61 () {60 vars in 1132 bytes} [Mon Aug 22 19:05:45 2022] POST /identity/v3/users => generated 249 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:05:46.040311 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c26eb31-87b1-4429-b3b1-6a3f0bbdedb3 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:46.040733 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c26eb31-87b1-4429-b3b1-6a3f0bbdedb3 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:46.041160 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c26eb31-87b1-4429-b3b1-6a3f0bbdedb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:46.061287 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-5c26eb31-87b1-4429-b3b1-6a3f0bbdedb3 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:05:46.063487 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-5c26eb31-87b1-4429-b3b1-6a3f0bbdedb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:05:46.148980 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 235/475] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:05:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 19:05:46.183623 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85078d59-abdc-4d9e-830b-a391549145f2 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:46.236334 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dd85a4-732d-4140-aa9d-e37fa3ae21d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:46.236962 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dd85a4-732d-4140-aa9d-e37fa3ae21d1 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:46.237245 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dd85a4-732d-4140-aa9d-e37fa3ae21d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:46.237522 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dd85a4-732d-4140-aa9d-e37fa3ae21d1 None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:46.238179 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dd85a4-732d-4140-aa9d-e37fa3ae21d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:46.238475 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dd85a4-732d-4140-aa9d-e37fa3ae21d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:46.240166 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dd85a4-732d-4140-aa9d-e37fa3ae21d1 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:46.254115 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 241/476] 10.222.0.61 () {56 vars in 1109 bytes} [Mon Aug 22 19:05:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:46.259599 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c26eb31-87b1-4429-b3b1-6a3f0bbdedb3 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:46.284063 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:46.370464 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c26eb31-87b1-4429-b3b1-6a3f0bbdedb3 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:46.406813 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b193674-ba77-481e-84fc-60cd2c26851a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:46.407350 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b193674-ba77-481e-84fc-60cd2c26851a None admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:46.407629 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b193674-ba77-481e-84fc-60cd2c26851a None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:46.407946 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b193674-ba77-481e-84fc-60cd2c26851a None admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:46.408603 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b193674-ba77-481e-84fc-60cd2c26851a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:46.410043 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b193674-ba77-481e-84fc-60cd2c26851a None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:46.433664 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 236/477] 10.222.0.61 () {60 vars in 1132 bytes} [Mon Aug 22 19:05:46 2022] POST /identity/v3/users => generated 248 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:05:46.484974 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dd85a4-732d-4140-aa9d-e37fa3ae21d1 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:46.567957 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:46.630988 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dd85a4-732d-4140-aa9d-e37fa3ae21d1 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:46.673805 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33425df4-0978-4cde-bd11-d9d127f3f6e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:46.674736 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33425df4-0978-4cde-bd11-d9d127f3f6e1 None admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:46.675234 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33425df4-0978-4cde-bd11-d9d127f3f6e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:46.675815 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33425df4-0978-4cde-bd11-d9d127f3f6e1 None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:46.676707 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33425df4-0978-4cde-bd11-d9d127f3f6e1 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:46.678290 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33425df4-0978-4cde-bd11-d9d127f3f6e1 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:46.691413 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.sql.core [None req-33425df4-0978-4cde-bd11-d9d127f3f6e1 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 22 19:05:46.691413 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 22 19:05:46.691413 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [parameters: {'id': 'e93473e7a81f4f82bd7a1fc8a19b584b', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 22 19:05:46.691413 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81533) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 22 19:05:46.711051 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-33425df4-0978-4cde-bd11-d9d127f3f6e1 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 22 19:05:46.711051 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:05:46.711051 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 22 19:05:46.711051 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 22 19:05:46.711051 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 22 19:05:46.711051 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 22 19:05:46.711051 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 22 19:05:46.711051 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application result = self._query(query) Aug 22 19:05:46.711051 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 22 19:05:46.711051 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application conn.query(q) Aug 22 19:05:46.711051 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 22 19:05:46.711051 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 22 19:05:46.711051 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 22 19:05:46.711051 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application result.read() Aug 22 19:05:46.711051 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 22 19:05:46.712007 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 22 19:05:46.712007 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 22 19:05:46.712007 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 22 19:05:46.712007 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 22 19:05:46.712007 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 22 19:05:46.712007 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 22 19:05:46.712007 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 22 19:05:46.712007 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 22 19:05:46.712007 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:05:46.712007 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 22 19:05:46.712007 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:05:46.712007 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:05:46.712007 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 22 19:05:46.712007 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 22 19:05:46.712007 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 22 19:05:46.712007 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return ref.to_dict() Aug 22 19:05:46.712007 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 22 19:05:46.712007 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application next(self.gen) Aug 22 19:05:46.712930 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 22 19:05:46.712930 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application yield resource Aug 22 19:05:46.712930 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 22 19:05:46.712930 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application next(self.gen) Aug 22 19:05:46.712930 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 22 19:05:46.712930 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self.session.rollback() Aug 22 19:05:46.712930 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 22 19:05:46.712930 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self.force_reraise() Aug 22 19:05:46.712930 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 22 19:05:46.712930 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise self.value Aug 22 19:05:46.712930 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 22 19:05:46.712930 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 22 19:05:46.712930 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 22 19:05:46.712930 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application session.commit() Aug 22 19:05:46.712930 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 22 19:05:46.712930 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 22 19:05:46.712930 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 22 19:05:46.713828 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self._prepare_impl() Aug 22 19:05:46.713828 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 22 19:05:46.713828 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self.session.flush() Aug 22 19:05:46.713828 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 22 19:05:46.713828 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self._flush(objects) Aug 22 19:05:46.713828 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 22 19:05:46.713828 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 22 19:05:46.713828 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 22 19:05:46.713828 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application compat.raise_( Aug 22 19:05:46.713828 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 22 19:05:46.713828 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception Aug 22 19:05:46.713828 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 22 19:05:46.713828 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application flush_context.execute() Aug 22 19:05:46.713828 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 22 19:05:46.713828 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rec.execute(self) Aug 22 19:05:46.713828 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 22 19:05:46.713828 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 22 19:05:46.714712 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 22 19:05:46.714712 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 22 19:05:46.714712 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 22 19:05:46.714712 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 22 19:05:46.714712 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 22 19:05:46.714712 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 22 19:05:46.714712 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 22 19:05:46.714712 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 22 19:05:46.714712 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 22 19:05:46.714712 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 22 19:05:46.714712 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 22 19:05:46.714712 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 22 19:05:46.714712 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 22 19:05:46.714712 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 22 19:05:46.714712 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 22 19:05:46.714712 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception Aug 22 19:05:46.715548 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 22 19:05:46.715548 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 22 19:05:46.715548 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 22 19:05:46.715548 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 22 19:05:46.715548 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 22 19:05:46.715548 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application result = self._query(query) Aug 22 19:05:46.715548 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 22 19:05:46.715548 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application conn.query(q) Aug 22 19:05:46.715548 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 22 19:05:46.715548 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 22 19:05:46.715548 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 22 19:05:46.715548 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application result.read() Aug 22 19:05:46.715548 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 22 19:05:46.715548 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 22 19:05:46.715548 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 22 19:05:46.715548 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 22 19:05:46.718380 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 22 19:05:46.718380 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 22 19:05:46.718380 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 22 19:05:46.718380 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 22 19:05:46.718380 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 22 19:05:46.718380 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 22 19:05:46.718380 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [parameters: {'id': 'e93473e7a81f4f82bd7a1fc8a19b584b', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 22 19:05:46.718380 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 22 19:05:46.718380 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:05:46.718380 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 19:05:46.718380 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:05:46.718380 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:05:46.718380 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:05:46.718380 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:05:46.718380 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:05:46.718380 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:05:46.721006 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:05:46.721006 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:05:46.721006 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:05:46.721006 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:05:46.721006 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:05:46.721006 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:05:46.721006 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 22 19:05:46.721006 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 22 19:05:46.721006 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:05:46.721006 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:05:46.721006 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 22 19:05:46.721006 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 22 19:05:46.721006 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 22 19:05:46.721006 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 22 19:05:46.721006 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 22 19:05:46.721006 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:05:46.724764 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 242/478] 10.222.0.61 () {60 vars in 1131 bytes} [Mon Aug 22 19:05:46 2022] POST /identity/v3/roles => generated 144 bytes in 240 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:05:46.730632 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b193674-ba77-481e-84fc-60cd2c26851a None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:46.821287 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:46.849899 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b193674-ba77-481e-84fc-60cd2c26851a None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:46.890165 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794430d0-2444-4839-b6e4-e3c83fccf9cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:46.891409 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794430d0-2444-4839-b6e4-e3c83fccf9cf None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:46.891710 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794430d0-2444-4839-b6e4-e3c83fccf9cf None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:46.891990 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794430d0-2444-4839-b6e4-e3c83fccf9cf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:46.907036 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794430d0-2444-4839-b6e4-e3c83fccf9cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 19:05:46.912914 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-794430d0-2444-4839-b6e4-e3c83fccf9cf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 19:05:46.912914 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:05:46.912914 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:05:46.912914 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:05:46.912914 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:05:46.912914 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:05:46.912914 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:05:46.912914 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:05:46.912914 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:05:46.912914 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:05:46.912914 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:05:46.912914 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:05:46.912914 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 19:05:46.912914 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 19:05:46.912914 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 19:05:46.912914 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 19:05:46.913785 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 19:05:46.913785 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 19:05:46.913785 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 19:05:46.913785 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 19:05:46.913785 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:05:46.913785 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:05:46.913785 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:05:46.913785 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:05:46.913785 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:05:46.913785 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:05:46.913785 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:05:46.913785 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:05:46.913785 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:05:46.913785 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:05:46.913785 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:05:46.913785 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:05:46.914593 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:05:46.914593 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:05:46.914593 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:05:46.914593 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:05:46.914593 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 19:05:46.914593 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 19:05:46.914593 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 19:05:46.914593 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 19:05:46.914593 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 19:05:46.914593 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 19:05:46.914593 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 19:05:46.914593 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:05:46.914593 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 237/479] 10.222.0.61 () {56 vars in 1114 bytes} [Mon Aug 22 19:05:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:05:46.919950 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33425df4-0978-4cde-bd11-d9d127f3f6e1 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:47.034764 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33425df4-0978-4cde-bd11-d9d127f3f6e1 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:47.055454 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:47.073655 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075c9515-20a5-42ac-ab73-3f42b4779682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:47.074265 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075c9515-20a5-42ac-ab73-3f42b4779682 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:47.074545 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075c9515-20a5-42ac-ab73-3f42b4779682 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:47.074820 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075c9515-20a5-42ac-ab73-3f42b4779682 None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:47.075491 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075c9515-20a5-42ac-ab73-3f42b4779682 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:47.075846 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075c9515-20a5-42ac-ab73-3f42b4779682 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:47.077223 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075c9515-20a5-42ac-ab73-3f42b4779682 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:47.090313 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 243/480] 10.222.0.61 () {56 vars in 1111 bytes} [Mon Aug 22 19:05:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:47.764064 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccb605f-60b1-4288-961a-65d6a7e5ea2f None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:47.764064 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccb605f-60b1-4288-961a-65d6a7e5ea2f None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:47.764064 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccb605f-60b1-4288-961a-65d6a7e5ea2f None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:47.765067 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 238/481] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:05:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:05:47.769144 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b9bd3c-d3d5-4f55-80ff-3e55e4a58286 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:47.769922 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b9bd3c-d3d5-4f55-80ff-3e55e4a58286 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:47.770372 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b9bd3c-d3d5-4f55-80ff-3e55e4a58286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:47.793951 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-08b9bd3c-d3d5-4f55-80ff-3e55e4a58286 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:05:47.795419 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-08b9bd3c-d3d5-4f55-80ff-3e55e4a58286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:05:47.881912 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 244/482] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:05:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 19:05:48.278116 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccb605f-60b1-4288-961a-65d6a7e5ea2f None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:48.315404 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75846d1-4548-491c-9911-531614afa9ed None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:48.315740 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75846d1-4548-491c-9911-531614afa9ed None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:48.315984 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75846d1-4548-491c-9911-531614afa9ed None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:48.317560 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 245/483] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:05:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:05:48.329756 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39784d2-15f9-40ec-a451-b2cfd53198b0 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:48.330034 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39784d2-15f9-40ec-a451-b2cfd53198b0 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:48.330288 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39784d2-15f9-40ec-a451-b2cfd53198b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:48.358513 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-a39784d2-15f9-40ec-a451-b2cfd53198b0 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:05:48.359776 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-a39784d2-15f9-40ec-a451-b2cfd53198b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:05:48.454421 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 246/484] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:05:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 22 19:05:48.474602 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccb605f-60b1-4288-961a-65d6a7e5ea2f None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:48.513592 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed597982-da9c-498a-9dbd-4055c5b14aee None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:48.513903 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed597982-da9c-498a-9dbd-4055c5b14aee None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:48.514151 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed597982-da9c-498a-9dbd-4055c5b14aee None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:48.516224 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 247/485] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:05:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 19:05:48.528645 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dd003d-336f-44ff-8aa5-e5ab577e1ba6 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:48.528933 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dd003d-336f-44ff-8aa5-e5ab577e1ba6 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:48.529201 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dd003d-336f-44ff-8aa5-e5ab577e1ba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:48.538992 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0766762c-3182-40f1-aabc-ef336c281315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:48.539983 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0766762c-3182-40f1-aabc-ef336c281315 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:48.540455 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0766762c-3182-40f1-aabc-ef336c281315 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:48.540913 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0766762c-3182-40f1-aabc-ef336c281315 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:48.547197 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-d0dd003d-336f-44ff-8aa5-e5ab577e1ba6 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:05:48.548429 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-d0dd003d-336f-44ff-8aa5-e5ab577e1ba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:05:48.550549 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:48.564733 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0766762c-3182-40f1-aabc-ef336c281315 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 19:05:48.569663 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-0766762c-3182-40f1-aabc-ef336c281315 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 19:05:48.569663 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:05:48.569663 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:05:48.569663 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:05:48.569663 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:05:48.569663 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:05:48.569663 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:05:48.569663 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:05:48.569663 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:05:48.569663 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:05:48.569663 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:05:48.569663 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:05:48.569663 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 19:05:48.569663 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 19:05:48.569663 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 19:05:48.569663 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 19:05:48.570565 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 19:05:48.570565 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 19:05:48.570565 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 19:05:48.570565 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 19:05:48.570565 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:05:48.570565 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:05:48.570565 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:05:48.570565 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:05:48.570565 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:05:48.570565 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:05:48.570565 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:05:48.570565 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:05:48.570565 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:05:48.570565 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:05:48.570565 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:05:48.570565 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:05:48.571360 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:05:48.571360 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:05:48.571360 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:05:48.571360 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:05:48.571360 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 19:05:48.571360 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 19:05:48.571360 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 19:05:48.571360 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 19:05:48.571360 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 19:05:48.571360 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 19:05:48.571360 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 19:05:48.571360 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:05:48.578750 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 239/486] 10.222.0.61 () {56 vars in 1102 bytes} [Mon Aug 22 19:05:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:05:48.584588 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0766762c-3182-40f1-aabc-ef336c281315 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:48.587856 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:48.626785 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 248/487] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:05:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 19:05:48.731985 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0766762c-3182-40f1-aabc-ef336c281315 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:48.775195 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b669e0ca-a6db-4f00-8662-f3f19bd8c9d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:48.775867 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b669e0ca-a6db-4f00-8662-f3f19bd8c9d7 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:48.776101 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b669e0ca-a6db-4f00-8662-f3f19bd8c9d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:48.776376 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b669e0ca-a6db-4f00-8662-f3f19bd8c9d7 None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:48.777052 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b669e0ca-a6db-4f00-8662-f3f19bd8c9d7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:48.777357 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b669e0ca-a6db-4f00-8662-f3f19bd8c9d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:48.778725 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b669e0ca-a6db-4f00-8662-f3f19bd8c9d7 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:48.790782 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 240/488] 10.222.0.61 () {56 vars in 1103 bytes} [Mon Aug 22 19:05:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:48.796205 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dd003d-336f-44ff-8aa5-e5ab577e1ba6 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:48.807617 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b669e0ca-a6db-4f00-8662-f3f19bd8c9d7 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:48.894563 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:48.939065 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b669e0ca-a6db-4f00-8662-f3f19bd8c9d7 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:48.962735 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dd003d-336f-44ff-8aa5-e5ab577e1ba6 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:48.985947 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649e8d0e-dc81-4c21-ac02-742f0170d0f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:48.986607 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649e8d0e-dc81-4c21-ac02-742f0170d0f1 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:48.986991 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649e8d0e-dc81-4c21-ac02-742f0170d0f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:48.987536 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649e8d0e-dc81-4c21-ac02-742f0170d0f1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:49.001686 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649e8d0e-dc81-4c21-ac02-742f0170d0f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 19:05:49.008000 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aea510c-6889-40cf-bfd8-7be7bb17e889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:49.008129 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-649e8d0e-dc81-4c21-ac02-742f0170d0f1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 19:05:49.008129 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:05:49.008129 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:05:49.008129 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:05:49.008129 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:05:49.008129 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:05:49.008129 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:05:49.008129 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:05:49.008129 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:05:49.008129 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:05:49.008129 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:05:49.008129 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:05:49.008129 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 19:05:49.008129 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 19:05:49.008129 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 19:05:49.008129 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 19:05:49.008992 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 19:05:49.008992 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 19:05:49.008992 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 19:05:49.008992 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 19:05:49.008992 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:05:49.008992 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:05:49.008992 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:05:49.008992 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:05:49.008992 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:05:49.008992 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:05:49.008992 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:05:49.008992 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:05:49.008992 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:05:49.008992 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:05:49.008992 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:05:49.008992 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:05:49.009790 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:05:49.009790 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:05:49.009790 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:05:49.009790 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:05:49.009790 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 19:05:49.009790 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 19:05:49.009790 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 19:05:49.009790 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 19:05:49.009790 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 19:05:49.009790 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 19:05:49.009790 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 19:05:49.009790 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:05:49.011561 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aea510c-6889-40cf-bfd8-7be7bb17e889 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:49.012084 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aea510c-6889-40cf-bfd8-7be7bb17e889 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:49.012554 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aea510c-6889-40cf-bfd8-7be7bb17e889 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:49.014128 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 241/489] 10.222.0.61 () {56 vars in 1102 bytes} [Mon Aug 22 19:05:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:05:49.015296 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649e8d0e-dc81-4c21-ac02-742f0170d0f1 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:49.025652 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aea510c-6889-40cf-bfd8-7be7bb17e889 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:49.027977 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aea510c-6889-40cf-bfd8-7be7bb17e889 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:49.053693 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-1aea510c-6889-40cf-bfd8-7be7bb17e889 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:05:49.053693 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:05:49.053693 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:05:49.053693 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:05:49.053693 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:05:49.053693 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:05:49.053693 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:05:49.053693 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:05:49.053693 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:05:49.053693 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:05:49.053693 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:05:49.053693 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:05:49.053693 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 19:05:49.053693 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 19:05:49.053693 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 19:05:49.053693 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 19:05:49.054638 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:05:49.054638 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:05:49.054638 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:05:49.054638 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:05:49.054638 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:05:49.054638 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:05:49.054638 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:05:49.054638 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:05:49.054638 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:05:49.054638 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:05:49.054638 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:05:49.054638 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:05:49.054638 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:05:49.054638 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:05:49.054638 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:05:49.054638 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:05:49.054638 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 19:05:49.055487 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 19:05:49.055487 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:05:49.055487 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:05:49.057039 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 249/490] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:05:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:05:49.060408 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aea510c-6889-40cf-bfd8-7be7bb17e889 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:49.092404 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:49.162025 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649e8d0e-dc81-4c21-ac02-742f0170d0f1 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:49.185460 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:49.195069 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aea510c-6889-40cf-bfd8-7be7bb17e889 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:49.207729 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1c9631-0255-4c86-9777-ca9f58e880e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:49.208845 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1c9631-0255-4c86-9777-ca9f58e880e0 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:49.209208 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1c9631-0255-4c86-9777-ca9f58e880e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:49.209574 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1c9631-0255-4c86-9777-ca9f58e880e0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:49.226031 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1c9631-0255-4c86-9777-ca9f58e880e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 19:05:49.231851 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-ac1c9631-0255-4c86-9777-ca9f58e880e0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 19:05:49.231851 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:05:49.231851 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:05:49.231851 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:05:49.231851 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:05:49.231851 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:05:49.231851 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:05:49.231851 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:05:49.231851 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:05:49.231851 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:05:49.231851 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:05:49.231851 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:05:49.231851 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 19:05:49.231851 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 19:05:49.231851 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 19:05:49.231851 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 19:05:49.232727 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 19:05:49.232727 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 19:05:49.232727 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 19:05:49.232727 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 19:05:49.232727 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:05:49.232727 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:05:49.232727 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:05:49.232727 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:05:49.232727 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:05:49.232727 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:05:49.232727 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:05:49.232727 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:05:49.232727 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:05:49.232727 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:05:49.232727 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:05:49.232727 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:05:49.233550 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:05:49.233550 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:05:49.233550 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:05:49.233550 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:05:49.233550 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 19:05:49.233550 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 19:05:49.233550 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 19:05:49.233550 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 19:05:49.233550 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 19:05:49.233550 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 19:05:49.233550 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 19:05:49.233550 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:05:49.238601 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 242/491] 10.222.0.61 () {56 vars in 1102 bytes} [Mon Aug 22 19:05:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:05:49.239873 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1c9631-0255-4c86-9777-ca9f58e880e0 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:49.245382 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39f1339-0647-4829-9a8d-a3685f6da5e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:49.246001 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39f1339-0647-4829-9a8d-a3685f6da5e1 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:49.246275 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39f1339-0647-4829-9a8d-a3685f6da5e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:49.246565 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39f1339-0647-4829-9a8d-a3685f6da5e1 None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:49.247234 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39f1339-0647-4829-9a8d-a3685f6da5e1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:49.247527 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39f1339-0647-4829-9a8d-a3685f6da5e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:49.248980 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39f1339-0647-4829-9a8d-a3685f6da5e1 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:49.260439 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 250/492] 10.222.0.61 () {56 vars in 1103 bytes} [Mon Aug 22 19:05:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:49.261081 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39f1339-0647-4829-9a8d-a3685f6da5e1 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:49.349729 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1c9631-0255-4c86-9777-ca9f58e880e0 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:49.352566 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:49.369678 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39f1339-0647-4829-9a8d-a3685f6da5e1 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:49.385463 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc117ef-3c75-4786-b419-c1bcd86c60be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:49.386072 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc117ef-3c75-4786-b419-c1bcd86c60be None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:49.386398 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc117ef-3c75-4786-b419-c1bcd86c60be None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:49.386609 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc117ef-3c75-4786-b419-c1bcd86c60be None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:49.387254 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc117ef-3c75-4786-b419-c1bcd86c60be None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:49.387550 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc117ef-3c75-4786-b419-c1bcd86c60be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:49.388980 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc117ef-3c75-4786-b419-c1bcd86c60be None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:49.405050 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe485b2-243f-4cfc-b8cd-96f569477198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:49.405664 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe485b2-243f-4cfc-b8cd-96f569477198 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:49.405964 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe485b2-243f-4cfc-b8cd-96f569477198 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:49.406774 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe485b2-243f-4cfc-b8cd-96f569477198 None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:49.407222 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 243/493] 10.222.0.61 () {56 vars in 1109 bytes} [Mon Aug 22 19:05:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:49.407441 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe485b2-243f-4cfc-b8cd-96f569477198 None admin] RBAC: Adding query filter params (name=service) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:49.407772 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe485b2-243f-4cfc-b8cd-96f569477198 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:49.408507 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58442021-da0f-42e2-bcda-607bc62b1ac7 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:49.408790 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58442021-da0f-42e2-bcda-607bc62b1ac7 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:49.409058 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58442021-da0f-42e2-bcda-607bc62b1ac7 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:49.409150 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe485b2-243f-4cfc-b8cd-96f569477198 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:49.410402 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 244/494] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:05:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:05:49.411025 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58442021-da0f-42e2-bcda-607bc62b1ac7 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:49.421748 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 251/495] 10.222.0.61 () {56 vars in 1103 bytes} [Mon Aug 22 19:05:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:49.422399 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe485b2-243f-4cfc-b8cd-96f569477198 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:49.458705 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:49.522230 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58442021-da0f-42e2-bcda-607bc62b1ac7 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:49.527054 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe485b2-243f-4cfc-b8cd-96f569477198 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:49.559758 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9919b2d4-d21e-4bfa-a8c0-a3520b1f6ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:49.560358 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9919b2d4-d21e-4bfa-a8c0-a3520b1f6ca2 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:49.560635 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9919b2d4-d21e-4bfa-a8c0-a3520b1f6ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:49.560901 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9919b2d4-d21e-4bfa-a8c0-a3520b1f6ca2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:49.561732 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a59b6ea-70c3-4ce4-bc5d-e755019d5e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:49.562391 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a59b6ea-70c3-4ce4-bc5d-e755019d5e10 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:49.562784 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a59b6ea-70c3-4ce4-bc5d-e755019d5e10 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:49.563053 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a59b6ea-70c3-4ce4-bc5d-e755019d5e10 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:49.570881 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a59b6ea-70c3-4ce4-bc5d-e755019d5e10 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:49.571009 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9919b2d4-d21e-4bfa-a8c0-a3520b1f6ca2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:49.572338 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a59b6ea-70c3-4ce4-bc5d-e755019d5e10 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:49.572443 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9919b2d4-d21e-4bfa-a8c0-a3520b1f6ca2 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:49.585685 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-7a59b6ea-70c3-4ce4-bc5d-e755019d5e10 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 22 19:05:49.585685 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:05:49.585685 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:05:49.585685 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:05:49.585685 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:05:49.585685 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:05:49.585685 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:05:49.585685 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:05:49.585685 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:05:49.585685 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:05:49.585685 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:05:49.585685 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:05:49.585685 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 19:05:49.585685 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 19:05:49.585685 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 19:05:49.585685 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 19:05:49.586633 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:05:49.586633 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:05:49.586633 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 19:05:49.586633 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 19:05:49.586633 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 19:05:49.586633 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 19:05:49.586633 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:05:49.586633 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:05:49.586633 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:05:49.586633 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:05:49.586633 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:05:49.586633 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:05:49.586633 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:05:49.586633 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:05:49.586633 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:05:49.586633 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:05:49.586633 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:05:49.586633 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:05:49.587549 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:05:49.587549 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:05:49.587549 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 19:05:49.587549 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 19:05:49.587549 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 19:05:49.587549 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 19:05:49.587549 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 19:05:49.587549 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 19:05:49.587549 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 22 19:05:49.587549 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:05:49.587549 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 252/496] 10.222.0.61 () {56 vars in 1093 bytes} [Mon Aug 22 19:05:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:05:49.588797 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb9d4af-9e1c-4c11-b33e-af70a47d228c None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:49.588797 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb9d4af-9e1c-4c11-b33e-af70a47d228c None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:49.588797 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb9d4af-9e1c-4c11-b33e-af70a47d228c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:49.591387 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-9919b2d4-d21e-4bfa-a8c0-a3520b1f6ca2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:05:49.591387 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:05:49.591387 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:05:49.591387 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:05:49.591387 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:05:49.591387 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:05:49.591387 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:05:49.591387 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:05:49.591387 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:05:49.591387 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:05:49.591387 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:05:49.591387 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:05:49.591387 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 19:05:49.591387 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 19:05:49.591387 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 19:05:49.591387 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 19:05:49.592895 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:05:49.592895 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:05:49.592895 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:05:49.592895 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:05:49.592895 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:05:49.592895 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:05:49.592895 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:05:49.592895 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:05:49.592895 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:05:49.592895 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:05:49.592895 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:05:49.592895 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:05:49.592895 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:05:49.592895 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:05:49.592895 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:05:49.592895 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:05:49.592895 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 19:05:49.595107 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 19:05:49.595107 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:05:49.595107 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:05:49.597639 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 245/497] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:05:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:05:49.597639 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9919b2d4-d21e-4bfa-a8c0-a3520b1f6ca2 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:49.612460 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-1cb9d4af-9e1c-4c11-b33e-af70a47d228c None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:05:49.614224 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-1cb9d4af-9e1c-4c11-b33e-af70a47d228c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:05:49.616330 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:49.681468 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 253/498] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:05:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 19:05:49.682424 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb9d4af-9e1c-4c11-b33e-af70a47d228c None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:49.701181 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:49.714438 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9919b2d4-d21e-4bfa-a8c0-a3520b1f6ca2 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:49.754152 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95328969-9073-43aa-9b78-ed4b5bb4bd53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:49.754862 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95328969-9073-43aa-9b78-ed4b5bb4bd53 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:49.755137 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95328969-9073-43aa-9b78-ed4b5bb4bd53 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:49.755408 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95328969-9073-43aa-9b78-ed4b5bb4bd53 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:49.766541 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95328969-9073-43aa-9b78-ed4b5bb4bd53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:49.768003 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95328969-9073-43aa-9b78-ed4b5bb4bd53 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:49.782462 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-95328969-9073-43aa-9b78-ed4b5bb4bd53 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:05:49.782462 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:05:49.782462 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:05:49.782462 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:05:49.782462 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:05:49.782462 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:05:49.782462 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:05:49.782462 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:05:49.782462 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:05:49.782462 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:05:49.782462 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:05:49.782462 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:05:49.782462 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 19:05:49.782462 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 19:05:49.782462 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 19:05:49.782462 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 19:05:49.784166 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:05:49.784166 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:05:49.784166 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:05:49.784166 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:05:49.784166 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:05:49.784166 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:05:49.784166 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:05:49.784166 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:05:49.784166 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:05:49.784166 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:05:49.784166 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:05:49.784166 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:05:49.784166 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:05:49.784166 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:05:49.784166 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:05:49.784166 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:05:49.784166 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 19:05:49.785045 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 19:05:49.785045 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:05:49.785045 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:05:49.785045 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 246/499] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:05:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:05:49.785045 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95328969-9073-43aa-9b78-ed4b5bb4bd53 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:49.794584 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb9d4af-9e1c-4c11-b33e-af70a47d228c None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:49.833166 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a1cfca-43fb-4bed-a446-f8131ddf7056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:49.833822 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a1cfca-43fb-4bed-a446-f8131ddf7056 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:49.834047 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a1cfca-43fb-4bed-a446-f8131ddf7056 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:49.834375 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a1cfca-43fb-4bed-a446-f8131ddf7056 None admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:49.835042 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a1cfca-43fb-4bed-a446-f8131ddf7056 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:49.835347 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a1cfca-43fb-4bed-a446-f8131ddf7056 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:49.837096 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a1cfca-43fb-4bed-a446-f8131ddf7056 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:49.856702 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:49.883406 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 254/500] 10.222.0.61 () {56 vars in 1133 bytes} [Mon Aug 22 19:05:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:49.883406 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e127bd40-98be-403c-a5af-343b1aa6306e None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:49.883406 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e127bd40-98be-403c-a5af-343b1aa6306e None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:49.883406 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e127bd40-98be-403c-a5af-343b1aa6306e None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:49.883406 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 255/501] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:05:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:05:49.883406 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e127bd40-98be-403c-a5af-343b1aa6306e None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:49.897148 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95328969-9073-43aa-9b78-ed4b5bb4bd53 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:49.930298 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecaf1e4-1023-4258-b77f-c0031c02ab4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:49.930888 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecaf1e4-1023-4258-b77f-c0031c02ab4a None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:49.931163 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecaf1e4-1023-4258-b77f-c0031c02ab4a None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:49.931428 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecaf1e4-1023-4258-b77f-c0031c02ab4a None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:49.932091 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecaf1e4-1023-4258-b77f-c0031c02ab4a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:49.932386 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecaf1e4-1023-4258-b77f-c0031c02ab4a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:49.933753 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecaf1e4-1023-4258-b77f-c0031c02ab4a None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:49.945843 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 247/502] 10.222.0.61 () {56 vars in 1109 bytes} [Mon Aug 22 19:05:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:49.946625 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecaf1e4-1023-4258-b77f-c0031c02ab4a None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:49.949258 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:49.995266 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e127bd40-98be-403c-a5af-343b1aa6306e None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:50.029882 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41629ba-0ead-46c7-b68d-facb266b3aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:50.030484 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41629ba-0ead-46c7-b68d-facb266b3aa5 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:50.030759 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41629ba-0ead-46c7-b68d-facb266b3aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:50.031056 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41629ba-0ead-46c7-b68d-facb266b3aa5 None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:50.031715 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41629ba-0ead-46c7-b68d-facb266b3aa5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:50.032014 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41629ba-0ead-46c7-b68d-facb266b3aa5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:50.033444 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41629ba-0ead-46c7-b68d-facb266b3aa5 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:50.044306 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 256/503] 10.222.0.61 () {56 vars in 1109 bytes} [Mon Aug 22 19:05:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:50.045576 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bba1ca-035d-4f16-a163-504b9f31103c None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:50.045814 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bba1ca-035d-4f16-a163-504b9f31103c None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:50.046084 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bba1ca-035d-4f16-a163-504b9f31103c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:50.047469 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecaf1e4-1023-4258-b77f-c0031c02ab4a None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:50.063623 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-c8bba1ca-035d-4f16-a163-504b9f31103c None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:05:50.064844 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-c8bba1ca-035d-4f16-a163-504b9f31103c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:05:50.082766 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c468c2-c6fe-4798-a838-845559fbeb5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:50.083400 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c468c2-c6fe-4798-a838-845559fbeb5b None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:50.083702 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c468c2-c6fe-4798-a838-845559fbeb5b None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:50.083980 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c468c2-c6fe-4798-a838-845559fbeb5b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:50.118149 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:50.135570 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 257/504] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:05:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 19:05:50.136410 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bba1ca-035d-4f16-a163-504b9f31103c None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:50.152112 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c468c2-c6fe-4798-a838-845559fbeb5b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:50.153584 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c468c2-c6fe-4798-a838-845559fbeb5b None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:50.155446 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:50.249075 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bba1ca-035d-4f16-a163-504b9f31103c None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:50.263897 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 248/505] 10.222.0.61 () {58 vars in 1282 bytes} [Mon Aug 22 19:05:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 19:05:50.263897 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c468c2-c6fe-4798-a838-845559fbeb5b None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:50.290692 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d432944-b4d2-45e4-b8b6-5b7a6525d902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:50.291284 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d432944-b4d2-45e4-b8b6-5b7a6525d902 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:50.291571 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d432944-b4d2-45e4-b8b6-5b7a6525d902 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:50.291877 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d432944-b4d2-45e4-b8b6-5b7a6525d902 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:50.299979 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d432944-b4d2-45e4-b8b6-5b7a6525d902 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:50.301437 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d432944-b4d2-45e4-b8b6-5b7a6525d902 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:50.314681 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-9d432944-b4d2-45e4-b8b6-5b7a6525d902 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 22 19:05:50.314681 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:05:50.314681 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:05:50.314681 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:05:50.314681 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:05:50.314681 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:05:50.314681 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:05:50.314681 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:05:50.314681 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:05:50.314681 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:05:50.314681 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:05:50.314681 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:05:50.314681 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 19:05:50.314681 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 19:05:50.314681 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 19:05:50.314681 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 19:05:50.315747 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:05:50.315747 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:05:50.315747 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 19:05:50.315747 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 19:05:50.315747 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 19:05:50.315747 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 19:05:50.315747 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:05:50.315747 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:05:50.315747 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:05:50.315747 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:05:50.315747 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:05:50.315747 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:05:50.315747 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:05:50.315747 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:05:50.315747 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:05:50.315747 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:05:50.315747 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:05:50.315747 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:05:50.316662 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:05:50.316662 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:05:50.316662 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 19:05:50.316662 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 19:05:50.316662 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 19:05:50.316662 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 19:05:50.316662 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 19:05:50.316662 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 19:05:50.316662 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 22 19:05:50.316662 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:05:50.316662 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 258/506] 10.222.0.61 () {56 vars in 1099 bytes} [Mon Aug 22 19:05:50 2022] GET /identity/v3/users/glance => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:05:50.316662 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d432944-b4d2-45e4-b8b6-5b7a6525d902 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:50.365261 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:50.366482 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c468c2-c6fe-4798-a838-845559fbeb5b None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:50.369198 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:50.402531 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31baaba-9a78-470a-a6b9-31301c7183d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:50.403157 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31baaba-9a78-470a-a6b9-31301c7183d9 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:50.403433 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31baaba-9a78-470a-a6b9-31301c7183d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:50.403729 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31baaba-9a78-470a-a6b9-31301c7183d9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:50.414138 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31baaba-9a78-470a-a6b9-31301c7183d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 19:05:50.418807 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-d31baaba-9a78-470a-a6b9-31301c7183d9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 19:05:50.418807 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:05:50.418807 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:05:50.418807 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:05:50.418807 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:05:50.418807 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:05:50.418807 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:05:50.418807 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:05:50.418807 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:05:50.418807 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:05:50.418807 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:05:50.418807 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:05:50.418807 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 19:05:50.418807 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 19:05:50.418807 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 19:05:50.418807 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 19:05:50.419756 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 19:05:50.419756 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 19:05:50.419756 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 19:05:50.419756 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 19:05:50.419756 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:05:50.419756 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:05:50.419756 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:05:50.419756 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:05:50.419756 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:05:50.419756 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:05:50.419756 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:05:50.419756 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:05:50.419756 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:05:50.419756 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:05:50.419756 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:05:50.419756 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:05:50.420569 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:05:50.420569 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:05:50.420569 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:05:50.420569 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:05:50.420569 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 19:05:50.420569 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 19:05:50.420569 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 19:05:50.420569 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 19:05:50.420569 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 19:05:50.420569 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 19:05:50.420569 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 19:05:50.420569 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:05:50.420569 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 249/507] 10.222.0.61 () {56 vars in 1102 bytes} [Mon Aug 22 19:05:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:05:50.421236 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31baaba-9a78-470a-a6b9-31301c7183d9 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:50.430016 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d432944-b4d2-45e4-b8b6-5b7a6525d902 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:50.463813 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1da818b-5d15-43ad-95dc-49cea466dab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:50.464440 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1da818b-5d15-43ad-95dc-49cea466dab2 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:50.464718 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1da818b-5d15-43ad-95dc-49cea466dab2 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:50.464985 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1da818b-5d15-43ad-95dc-49cea466dab2 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:50.472954 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1da818b-5d15-43ad-95dc-49cea466dab2 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:50.474397 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1da818b-5d15-43ad-95dc-49cea466dab2 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:50.485973 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-c1da818b-5d15-43ad-95dc-49cea466dab2 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 22 19:05:50.485973 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:05:50.485973 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:05:50.485973 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:05:50.485973 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:05:50.485973 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:05:50.485973 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:05:50.485973 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:05:50.485973 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:05:50.485973 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:05:50.485973 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:05:50.485973 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:05:50.485973 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 19:05:50.485973 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 19:05:50.485973 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 19:05:50.485973 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 19:05:50.487066 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:05:50.487066 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:05:50.487066 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 19:05:50.487066 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 19:05:50.487066 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 19:05:50.487066 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 19:05:50.487066 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:05:50.487066 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:05:50.487066 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:05:50.487066 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:05:50.487066 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:05:50.487066 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:05:50.487066 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:05:50.487066 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:05:50.487066 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:05:50.487066 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:05:50.487066 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:05:50.487066 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:05:50.488190 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:05:50.488190 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:05:50.488190 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 19:05:50.488190 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 19:05:50.488190 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 19:05:50.488190 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 19:05:50.488190 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 19:05:50.488190 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 19:05:50.488190 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 22 19:05:50.488190 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:05:50.488190 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 259/508] 10.222.0.61 () {56 vars in 1102 bytes} [Mon Aug 22 19:05:50 2022] GET /identity/v3/users/neutron => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:05:50.488190 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1da818b-5d15-43ad-95dc-49cea466dab2 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:50.527667 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31baaba-9a78-470a-a6b9-31301c7183d9 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:50.562159 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053b1a85-8b8e-43cb-bb26-7a9e949d9580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:50.563149 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053b1a85-8b8e-43cb-bb26-7a9e949d9580 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:50.563431 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053b1a85-8b8e-43cb-bb26-7a9e949d9580 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:50.563738 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053b1a85-8b8e-43cb-bb26-7a9e949d9580 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:50.574378 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053b1a85-8b8e-43cb-bb26-7a9e949d9580 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:50.575854 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053b1a85-8b8e-43cb-bb26-7a9e949d9580 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:50.586793 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1da818b-5d15-43ad-95dc-49cea466dab2 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:50.590101 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-053b1a85-8b8e-43cb-bb26-7a9e949d9580 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:05:50.590101 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:05:50.590101 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:05:50.590101 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:05:50.590101 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:05:50.590101 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:05:50.590101 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:05:50.590101 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:05:50.590101 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:05:50.590101 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:05:50.590101 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:05:50.590101 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:05:50.590101 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 19:05:50.590101 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 19:05:50.590101 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 19:05:50.590101 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 19:05:50.591160 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:05:50.591160 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:05:50.591160 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:05:50.591160 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:05:50.591160 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:05:50.591160 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:05:50.591160 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:05:50.591160 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:05:50.591160 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:05:50.591160 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:05:50.591160 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:05:50.591160 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:05:50.591160 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:05:50.591160 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:05:50.591160 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:05:50.591160 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:05:50.591160 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 19:05:50.592267 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 19:05:50.592267 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:05:50.592267 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:05:50.592267 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 250/509] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:05:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:05:50.592267 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053b1a85-8b8e-43cb-bb26-7a9e949d9580 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:50.597615 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:50.608118 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:50.625870 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b536152-498a-43bf-9b95-14dead35f894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:50.626469 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b536152-498a-43bf-9b95-14dead35f894 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:50.626746 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b536152-498a-43bf-9b95-14dead35f894 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:50.627010 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b536152-498a-43bf-9b95-14dead35f894 None admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:50.627773 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b536152-498a-43bf-9b95-14dead35f894 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:50.628075 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b536152-498a-43bf-9b95-14dead35f894 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:50.629495 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b536152-498a-43bf-9b95-14dead35f894 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:50.646341 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 260/510] 10.222.0.61 () {56 vars in 1137 bytes} [Mon Aug 22 19:05:50 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:50.647197 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b536152-498a-43bf-9b95-14dead35f894 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:50.694098 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053b1a85-8b8e-43cb-bb26-7a9e949d9580 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:50.728100 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2aa114-2c4c-4ae2-9ca7-d6fc5429c2a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:50.728717 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2aa114-2c4c-4ae2-9ca7-d6fc5429c2a4 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:50.728992 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2aa114-2c4c-4ae2-9ca7-d6fc5429c2a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:50.729260 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2aa114-2c4c-4ae2-9ca7-d6fc5429c2a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:50.740837 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2aa114-2c4c-4ae2-9ca7-d6fc5429c2a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:50.742270 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2aa114-2c4c-4ae2-9ca7-d6fc5429c2a4 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:50.743263 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b536152-498a-43bf-9b95-14dead35f894 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:50.755739 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-6f2aa114-2c4c-4ae2-9ca7-d6fc5429c2a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:05:50.755739 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:05:50.755739 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:05:50.755739 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:05:50.755739 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:05:50.755739 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:05:50.755739 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:05:50.755739 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:05:50.755739 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:05:50.755739 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:05:50.755739 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:05:50.755739 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:05:50.755739 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 19:05:50.755739 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 19:05:50.755739 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 19:05:50.755739 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 19:05:50.756797 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:05:50.756797 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:05:50.756797 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:05:50.756797 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:05:50.756797 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:05:50.756797 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:05:50.756797 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:05:50.756797 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:05:50.756797 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:05:50.756797 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:05:50.756797 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:05:50.756797 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:05:50.756797 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:05:50.756797 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:05:50.756797 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:05:50.756797 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:05:50.756797 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 19:05:50.757730 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 19:05:50.757730 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:05:50.757730 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:05:50.757730 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 251/511] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:05:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:05:50.757730 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2aa114-2c4c-4ae2-9ca7-d6fc5429c2a4 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:50.779513 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01c8cc3-8efa-43ef-a08d-febcce28f7d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:50.780263 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01c8cc3-8efa-43ef-a08d-febcce28f7d2 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:50.780539 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01c8cc3-8efa-43ef-a08d-febcce28f7d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:50.780809 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01c8cc3-8efa-43ef-a08d-febcce28f7d2 None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:50.781974 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01c8cc3-8efa-43ef-a08d-febcce28f7d2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:50.782278 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01c8cc3-8efa-43ef-a08d-febcce28f7d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:50.783659 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01c8cc3-8efa-43ef-a08d-febcce28f7d2 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:50.791342 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 261/512] 10.222.0.61 () {56 vars in 1103 bytes} [Mon Aug 22 19:05:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:50.792135 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01c8cc3-8efa-43ef-a08d-febcce28f7d2 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:50.819880 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:50.823156 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:50.858573 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2aa114-2c4c-4ae2-9ca7-d6fc5429c2a4 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:50.891001 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76db1758-82a5-4aa9-bd4d-e9493c4c0091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:50.891596 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76db1758-82a5-4aa9-bd4d-e9493c4c0091 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:50.891903 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76db1758-82a5-4aa9-bd4d-e9493c4c0091 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:50.892195 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76db1758-82a5-4aa9-bd4d-e9493c4c0091 None admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:50.892487 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01c8cc3-8efa-43ef-a08d-febcce28f7d2 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:50.892923 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76db1758-82a5-4aa9-bd4d-e9493c4c0091 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:50.893227 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76db1758-82a5-4aa9-bd4d-e9493c4c0091 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:50.894660 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76db1758-82a5-4aa9-bd4d-e9493c4c0091 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:50.926274 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe8e0dd-e1f4-496e-8c50-16d4b2b183ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:50.926878 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe8e0dd-e1f4-496e-8c50-16d4b2b183ac None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:50.927152 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe8e0dd-e1f4-496e-8c50-16d4b2b183ac None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:50.927422 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe8e0dd-e1f4-496e-8c50-16d4b2b183ac None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:50.928079 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe8e0dd-e1f4-496e-8c50-16d4b2b183ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:50.928377 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe8e0dd-e1f4-496e-8c50-16d4b2b183ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:50.929753 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe8e0dd-e1f4-496e-8c50-16d4b2b183ac None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:50.936211 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 252/513] 10.222.0.61 () {56 vars in 1139 bytes} [Mon Aug 22 19:05:50 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:50.937045 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76db1758-82a5-4aa9-bd4d-e9493c4c0091 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:50.948987 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 262/514] 10.222.0.61 () {56 vars in 1109 bytes} [Mon Aug 22 19:05:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:50.948987 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe8e0dd-e1f4-496e-8c50-16d4b2b183ac None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:51.032057 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:51.042728 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76db1758-82a5-4aa9-bd4d-e9493c4c0091 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:51.047110 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe8e0dd-e1f4-496e-8c50-16d4b2b183ac None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:51.066116 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:51.077026 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb9cb37-f496-4f3d-aa0e-b9096dee952c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:51.077575 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb9cb37-f496-4f3d-aa0e-b9096dee952c None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:51.078387 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb9cb37-f496-4f3d-aa0e-b9096dee952c None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:51.078660 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb9cb37-f496-4f3d-aa0e-b9096dee952c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:51.080734 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74947060-3412-429e-bd4f-9c9925243171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:51.081332 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74947060-3412-429e-bd4f-9c9925243171 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:51.081601 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74947060-3412-429e-bd4f-9c9925243171 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:51.081871 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74947060-3412-429e-bd4f-9c9925243171 None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:51.082508 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74947060-3412-429e-bd4f-9c9925243171 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:51.082803 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74947060-3412-429e-bd4f-9c9925243171 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:51.084232 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74947060-3412-429e-bd4f-9c9925243171 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:51.095955 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 263/515] 10.222.0.61 () {56 vars in 1109 bytes} [Mon Aug 22 19:05:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:51.096782 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74947060-3412-429e-bd4f-9c9925243171 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:51.149647 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb9cb37-f496-4f3d-aa0e-b9096dee952c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:51.151107 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb9cb37-f496-4f3d-aa0e-b9096dee952c None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:51.196126 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74947060-3412-429e-bd4f-9c9925243171 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:51.229134 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87d6fa0-81ab-43c0-af33-63e48b485f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:51.229892 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87d6fa0-81ab-43c0-af33-63e48b485f23 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:51.230264 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87d6fa0-81ab-43c0-af33-63e48b485f23 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:51.230638 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87d6fa0-81ab-43c0-af33-63e48b485f23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:51.241884 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87d6fa0-81ab-43c0-af33-63e48b485f23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:51.243324 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87d6fa0-81ab-43c0-af33-63e48b485f23 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:51.247569 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 253/516] 10.222.0.61 () {58 vars in 1282 bytes} [Mon Aug 22 19:05:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 19:05:51.248420 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb9cb37-f496-4f3d-aa0e-b9096dee952c None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:51.250521 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:51.259415 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-d87d6fa0-81ab-43c0-af33-63e48b485f23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:05:51.259415 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:05:51.259415 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:05:51.259415 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:05:51.259415 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:05:51.259415 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:05:51.259415 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:05:51.259415 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:05:51.259415 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:05:51.259415 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:05:51.259415 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:05:51.259415 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:05:51.259415 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 19:05:51.259415 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 19:05:51.259415 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 19:05:51.259415 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 19:05:51.260531 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:05:51.260531 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:05:51.260531 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:05:51.260531 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:05:51.260531 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:05:51.260531 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:05:51.260531 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:05:51.260531 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:05:51.260531 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:05:51.260531 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:05:51.260531 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:05:51.260531 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:05:51.260531 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:05:51.260531 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:05:51.260531 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:05:51.260531 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:05:51.260531 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 19:05:51.261388 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 19:05:51.261388 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:05:51.261388 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:05:51.261388 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 264/517] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:05:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:05:51.261643 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87d6fa0-81ab-43c0-af33-63e48b485f23 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:51.278848 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:51.352098 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb9cb37-f496-4f3d-aa0e-b9096dee952c None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:51.361426 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87d6fa0-81ab-43c0-af33-63e48b485f23 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:51.385116 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a89c0a6-921b-4d59-83a7-1d6aa22168ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:51.385708 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a89c0a6-921b-4d59-83a7-1d6aa22168ba None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:51.385982 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a89c0a6-921b-4d59-83a7-1d6aa22168ba None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:51.386329 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a89c0a6-921b-4d59-83a7-1d6aa22168ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:51.394758 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096e3ea2-b339-4e5e-97ea-e8ae31f26c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:51.395375 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096e3ea2-b339-4e5e-97ea-e8ae31f26c51 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:51.395676 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096e3ea2-b339-4e5e-97ea-e8ae31f26c51 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:51.395938 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096e3ea2-b339-4e5e-97ea-e8ae31f26c51 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:51.402387 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096e3ea2-b339-4e5e-97ea-e8ae31f26c51 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:51.403841 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096e3ea2-b339-4e5e-97ea-e8ae31f26c51 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:51.415547 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-096e3ea2-b339-4e5e-97ea-e8ae31f26c51 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 19:05:51.415547 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:05:51.415547 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:05:51.415547 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:05:51.415547 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:05:51.415547 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:05:51.415547 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:05:51.415547 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:05:51.415547 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:05:51.415547 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:05:51.415547 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:05:51.415547 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:05:51.415547 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 19:05:51.415547 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 19:05:51.415547 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 19:05:51.416616 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 19:05:51.416616 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:05:51.416616 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:05:51.416616 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:05:51.416616 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:05:51.416616 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:05:51.416616 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:05:51.416616 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:05:51.416616 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:05:51.416616 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:05:51.416616 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:05:51.416616 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:05:51.416616 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:05:51.416616 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:05:51.416616 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:05:51.416616 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:05:51.416616 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:05:51.417686 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 19:05:51.417686 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 19:05:51.417686 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 19:05:51.417686 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 19:05:51.417686 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 19:05:51.417686 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 19:05:51.417686 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 19:05:51.417686 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:05:51.417686 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 265/518] 10.222.0.61 () {56 vars in 1111 bytes} [Mon Aug 22 19:05:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:05:51.417686 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096e3ea2-b339-4e5e-97ea-e8ae31f26c51 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:51.457692 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a89c0a6-921b-4d59-83a7-1d6aa22168ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:51.459136 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a89c0a6-921b-4d59-83a7-1d6aa22168ba None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:51.474922 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:51.482334 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:51.519474 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096e3ea2-b339-4e5e-97ea-e8ae31f26c51 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:51.552357 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3162fdb4-2ed8-407e-b3cc-7aaaf3589aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:51.552974 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3162fdb4-2ed8-407e-b3cc-7aaaf3589aa2 None admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:51.553251 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3162fdb4-2ed8-407e-b3cc-7aaaf3589aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:51.553517 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3162fdb4-2ed8-407e-b3cc-7aaaf3589aa2 None admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:51.554139 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3162fdb4-2ed8-407e-b3cc-7aaaf3589aa2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:51.555543 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3162fdb4-2ed8-407e-b3cc-7aaaf3589aa2 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:51.557285 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 254/519] 10.222.0.61 () {58 vars in 1282 bytes} [Mon Aug 22 19:05:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 19:05:51.558105 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a89c0a6-921b-4d59-83a7-1d6aa22168ba None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:51.577253 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 266/520] 10.222.0.61 () {60 vars in 1132 bytes} [Mon Aug 22 19:05:51 2022] POST /identity/v3/users => generated 247 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:05:51.588790 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3162fdb4-2ed8-407e-b3cc-7aaaf3589aa2 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:51.667605 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a89c0a6-921b-4d59-83a7-1d6aa22168ba None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:51.693635 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:51.703602 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3162fdb4-2ed8-407e-b3cc-7aaaf3589aa2 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:51.705970 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87a04a4-88ba-409f-9ea5-1852b98105d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:51.706596 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87a04a4-88ba-409f-9ea5-1852b98105d5 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:51.706872 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87a04a4-88ba-409f-9ea5-1852b98105d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:51.707134 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87a04a4-88ba-409f-9ea5-1852b98105d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:51.718654 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87a04a4-88ba-409f-9ea5-1852b98105d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:51.719665 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:51.720137 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87a04a4-88ba-409f-9ea5-1852b98105d5 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:51.736707 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-b87a04a4-88ba-409f-9ea5-1852b98105d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:05:51.736707 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:05:51.736707 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:05:51.736707 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:05:51.736707 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:05:51.736707 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:05:51.736707 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:05:51.736707 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:05:51.736707 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:05:51.736707 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:05:51.736707 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:05:51.736707 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:05:51.736707 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 19:05:51.736707 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 19:05:51.736707 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 19:05:51.736707 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 19:05:51.737825 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:05:51.737825 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:05:51.737825 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:05:51.737825 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:05:51.737825 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:05:51.737825 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:05:51.737825 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:05:51.737825 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:05:51.737825 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:05:51.737825 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:05:51.737825 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:05:51.737825 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:05:51.737825 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:05:51.737825 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:05:51.737825 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:05:51.737825 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:05:51.737825 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 19:05:51.738819 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 19:05:51.738819 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:05:51.738819 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:05:51.738819 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 255/521] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:05:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:05:51.738819 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87a04a4-88ba-409f-9ea5-1852b98105d5 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:51.748408 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026a6ef0-df63-4fe0-a9bd-5267d8c7de11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:51.749027 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026a6ef0-df63-4fe0-a9bd-5267d8c7de11 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:51.749307 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026a6ef0-df63-4fe0-a9bd-5267d8c7de11 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:51.749581 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026a6ef0-df63-4fe0-a9bd-5267d8c7de11 None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:51.750808 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026a6ef0-df63-4fe0-a9bd-5267d8c7de11 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:51.751111 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026a6ef0-df63-4fe0-a9bd-5267d8c7de11 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:51.752527 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026a6ef0-df63-4fe0-a9bd-5267d8c7de11 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:51.766013 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 267/522] 10.222.0.61 () {56 vars in 1109 bytes} [Mon Aug 22 19:05:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:51.766843 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026a6ef0-df63-4fe0-a9bd-5267d8c7de11 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:51.853909 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87a04a4-88ba-409f-9ea5-1852b98105d5 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:51.883244 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026a6ef0-df63-4fe0-a9bd-5267d8c7de11 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:51.888405 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707d4107-d902-468c-acb0-b91574590f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:51.889013 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707d4107-d902-468c-acb0-b91574590f6c None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:51.889286 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707d4107-d902-468c-acb0-b91574590f6c None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:51.889554 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707d4107-d902-468c-acb0-b91574590f6c None admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:51.890233 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707d4107-d902-468c-acb0-b91574590f6c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:51.891366 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707d4107-d902-468c-acb0-b91574590f6c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:51.892528 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707d4107-d902-468c-acb0-b91574590f6c None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:51.910513 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 256/523] 10.222.0.61 () {56 vars in 1148 bytes} [Mon Aug 22 19:05:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:51.911336 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707d4107-d902-468c-acb0-b91574590f6c None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:51.926350 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b43c74c-1842-4de5-8c3f-2bf6c2f14f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:51.926949 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b43c74c-1842-4de5-8c3f-2bf6c2f14f1d None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:51.927183 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b43c74c-1842-4de5-8c3f-2bf6c2f14f1d None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:51.927450 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b43c74c-1842-4de5-8c3f-2bf6c2f14f1d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:51.937791 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b43c74c-1842-4de5-8c3f-2bf6c2f14f1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:51.939242 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b43c74c-1842-4de5-8c3f-2bf6c2f14f1d None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:51.949218 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:51.954856 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-1b43c74c-1842-4de5-8c3f-2bf6c2f14f1d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:05:51.954856 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:05:51.954856 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:05:51.954856 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:05:51.954856 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:05:51.954856 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:05:51.954856 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:05:51.954856 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:05:51.954856 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:05:51.954856 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:05:51.954856 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:05:51.954856 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:05:51.954856 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 19:05:51.954856 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 19:05:51.954856 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 19:05:51.954856 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 19:05:51.955810 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:05:51.955810 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:05:51.955810 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:05:51.955810 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:05:51.955810 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:05:51.955810 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:05:51.955810 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:05:51.955810 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:05:51.955810 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:05:51.955810 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:05:51.955810 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:05:51.955810 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:05:51.955810 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:05:51.955810 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:05:51.955810 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:05:51.955810 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:05:51.955810 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 19:05:51.956816 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 19:05:51.956816 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:05:51.956816 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:05:51.956816 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 268/524] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:05:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:05:51.956816 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b43c74c-1842-4de5-8c3f-2bf6c2f14f1d None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:51.994652 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:52.023876 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707d4107-d902-468c-acb0-b91574590f6c None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:52.058439 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d042c7d3-06f4-4a23-8c85-60d6b0c92136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:52.059047 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d042c7d3-06f4-4a23-8c85-60d6b0c92136 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:52.059329 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d042c7d3-06f4-4a23-8c85-60d6b0c92136 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:52.059595 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d042c7d3-06f4-4a23-8c85-60d6b0c92136 None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:52.060271 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d042c7d3-06f4-4a23-8c85-60d6b0c92136 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:52.060569 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d042c7d3-06f4-4a23-8c85-60d6b0c92136 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:52.061938 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d042c7d3-06f4-4a23-8c85-60d6b0c92136 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:52.074390 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 257/525] 10.222.0.61 () {56 vars in 1109 bytes} [Mon Aug 22 19:05:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:52.075011 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d042c7d3-06f4-4a23-8c85-60d6b0c92136 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:52.081516 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b43c74c-1842-4de5-8c3f-2bf6c2f14f1d None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:52.119134 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8a1630-ce85-4fc0-9075-a4053702a3e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:52.119839 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8a1630-ce85-4fc0-9075-a4053702a3e8 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:52.120114 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8a1630-ce85-4fc0-9075-a4053702a3e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:52.120380 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8a1630-ce85-4fc0-9075-a4053702a3e8 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:52.129576 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8a1630-ce85-4fc0-9075-a4053702a3e8 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:52.131198 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8a1630-ce85-4fc0-9075-a4053702a3e8 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:52.143712 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-dd8a1630-ce85-4fc0-9075-a4053702a3e8 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 22 19:05:52.143712 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:05:52.143712 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:05:52.143712 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:05:52.143712 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:05:52.143712 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:05:52.143712 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:05:52.143712 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:05:52.143712 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:05:52.143712 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:05:52.143712 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:05:52.143712 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:05:52.143712 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 19:05:52.143712 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 19:05:52.143712 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 19:05:52.143712 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 19:05:52.144668 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:05:52.144668 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:05:52.144668 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 19:05:52.144668 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 19:05:52.144668 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 19:05:52.144668 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 19:05:52.144668 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:05:52.144668 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:05:52.144668 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:05:52.144668 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:05:52.144668 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:05:52.144668 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:05:52.144668 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:05:52.144668 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:05:52.144668 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:05:52.144668 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:05:52.144668 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:05:52.144668 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:05:52.146297 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:05:52.146297 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:05:52.146297 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 19:05:52.146297 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 19:05:52.146297 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 19:05:52.146297 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 19:05:52.146297 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 19:05:52.146297 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 19:05:52.146297 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 22 19:05:52.146297 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:05:52.146297 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 269/526] 10.222.0.61 () {56 vars in 1099 bytes} [Mon Aug 22 19:05:51 2022] GET /identity/v3/users/cinder => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:05:52.146297 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8a1630-ce85-4fc0-9075-a4053702a3e8 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:52.185069 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:52.191819 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d042c7d3-06f4-4a23-8c85-60d6b0c92136 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:52.227243 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ee2ab1-24db-41fa-b561-cae1fc7ebaaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:52.228119 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ee2ab1-24db-41fa-b561-cae1fc7ebaaa None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:52.228119 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ee2ab1-24db-41fa-b561-cae1fc7ebaaa None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:52.228399 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ee2ab1-24db-41fa-b561-cae1fc7ebaaa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:52.229118 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ee2ab1-24db-41fa-b561-cae1fc7ebaaa None admin] RBAC: Adding query filter params (role.id=324bd04d55c34b61a5282c9f6ed73f46, scope.project.id=e07dc5ebdc504c6da706a2d21c0a3970, user.id=e7f43e2756b746a6825ca1a06b7d7c91) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:52.229419 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ee2ab1-24db-41fa-b561-cae1fc7ebaaa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:52.230841 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ee2ab1-24db-41fa-b561-cae1fc7ebaaa None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:52.236648 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 258/527] 10.222.0.61 () {56 vars in 1374 bytes} [Mon Aug 22 19:05:52 2022] GET /identity/v3/role_assignments?user.id=e7f43e2756b746a6825ca1a06b7d7c91&scope.project.id=e07dc5ebdc504c6da706a2d21c0a3970&role.id=324bd04d55c34b61a5282c9f6ed73f46 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:52.237337 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ee2ab1-24db-41fa-b561-cae1fc7ebaaa None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:52.250980 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:52.277327 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8a1630-ce85-4fc0-9075-a4053702a3e8 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:52.318563 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3a50f6-e8eb-4a9d-a7a6-40ee7325e6a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:52.319269 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3a50f6-e8eb-4a9d-a7a6-40ee7325e6a3 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:52.319836 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3a50f6-e8eb-4a9d-a7a6-40ee7325e6a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:52.319836 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3a50f6-e8eb-4a9d-a7a6-40ee7325e6a3 None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:52.320549 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3a50f6-e8eb-4a9d-a7a6-40ee7325e6a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:52.320853 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3a50f6-e8eb-4a9d-a7a6-40ee7325e6a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:52.322245 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3a50f6-e8eb-4a9d-a7a6-40ee7325e6a3 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:52.341161 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 270/528] 10.222.0.61 () {56 vars in 1109 bytes} [Mon Aug 22 19:05:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:52.342168 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3a50f6-e8eb-4a9d-a7a6-40ee7325e6a3 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:52.375930 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ee2ab1-24db-41fa-b561-cae1fc7ebaaa None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:52.419426 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f465a815-8dd1-4ac7-9189-ad90b94ed940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:52.420099 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f465a815-8dd1-4ac7-9189-ad90b94ed940 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:52.420375 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f465a815-8dd1-4ac7-9189-ad90b94ed940 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:52.420643 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f465a815-8dd1-4ac7-9189-ad90b94ed940 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:52.430405 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f465a815-8dd1-4ac7-9189-ad90b94ed940 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:52.431908 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f465a815-8dd1-4ac7-9189-ad90b94ed940 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:52.448359 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-f465a815-8dd1-4ac7-9189-ad90b94ed940 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 19:05:52.448359 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:05:52.448359 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:05:52.448359 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:05:52.448359 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:05:52.448359 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:05:52.448359 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:05:52.448359 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:05:52.448359 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:05:52.448359 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:05:52.448359 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:05:52.448359 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:05:52.448359 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 19:05:52.448359 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 19:05:52.448359 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 19:05:52.449324 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 19:05:52.449324 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:05:52.449324 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:05:52.449324 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:05:52.449324 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:05:52.449324 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:05:52.449324 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:05:52.449324 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:05:52.449324 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:05:52.449324 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:05:52.449324 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:05:52.449324 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:05:52.449324 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:05:52.449324 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:05:52.449324 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:05:52.449324 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:05:52.449324 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:05:52.450175 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 19:05:52.450175 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 19:05:52.450175 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 19:05:52.450175 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 19:05:52.450175 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 19:05:52.450175 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 19:05:52.450175 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 19:05:52.450175 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:05:52.454323 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 259/529] 10.222.0.61 () {56 vars in 1111 bytes} [Mon Aug 22 19:05:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:05:52.455361 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f465a815-8dd1-4ac7-9189-ad90b94ed940 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:52.469176 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3a50f6-e8eb-4a9d-a7a6-40ee7325e6a3 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:52.483011 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:52.510507 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176b1546-3795-4420-9ff4-9d6ed2c87d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:52.511114 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176b1546-3795-4420-9ff4-9d6ed2c87d41 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:52.511390 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176b1546-3795-4420-9ff4-9d6ed2c87d41 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:52.511677 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176b1546-3795-4420-9ff4-9d6ed2c87d41 None admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:52.512448 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176b1546-3795-4420-9ff4-9d6ed2c87d41 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:52.512751 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176b1546-3795-4420-9ff4-9d6ed2c87d41 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:52.514339 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176b1546-3795-4420-9ff4-9d6ed2c87d41 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:52.535689 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 271/530] 10.222.0.61 () {56 vars in 1137 bytes} [Mon Aug 22 19:05:52 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:52.535689 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176b1546-3795-4420-9ff4-9d6ed2c87d41 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:52.542762 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:52.587523 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f465a815-8dd1-4ac7-9189-ad90b94ed940 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:52.625315 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d191aca-e8e6-41b9-a989-a60f52f1d9fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:52.626178 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d191aca-e8e6-41b9-a989-a60f52f1d9fa None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:52.626598 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d191aca-e8e6-41b9-a989-a60f52f1d9fa None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:52.627038 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d191aca-e8e6-41b9-a989-a60f52f1d9fa None admin] PATH_INFO: `/v3/projects/service` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:52.634565 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d191aca-e8e6-41b9-a989-a60f52f1d9fa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:52.636310 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d191aca-e8e6-41b9-a989-a60f52f1d9fa None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:52.648885 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176b1546-3795-4420-9ff4-9d6ed2c87d41 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:52.649333 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-5d191aca-e8e6-41b9-a989-a60f52f1d9fa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 19:05:52.649333 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:05:52.649333 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:05:52.649333 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:05:52.649333 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:05:52.649333 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:05:52.649333 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:05:52.649333 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:05:52.649333 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:05:52.649333 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:05:52.649333 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:05:52.649333 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:05:52.649333 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 19:05:52.649333 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 19:05:52.649333 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 19:05:52.650341 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 19:05:52.650341 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:05:52.650341 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:05:52.650341 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:05:52.650341 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:05:52.650341 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:05:52.650341 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:05:52.650341 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:05:52.650341 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:05:52.650341 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:05:52.650341 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:05:52.650341 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:05:52.650341 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:05:52.650341 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:05:52.650341 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:05:52.650341 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:05:52.650341 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:05:52.651390 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 19:05:52.651390 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 19:05:52.651390 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 19:05:52.651390 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 19:05:52.651390 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 19:05:52.651390 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 19:05:52.651390 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 19:05:52.651390 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:05:52.653491 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 260/531] 10.222.0.61 () {56 vars in 1111 bytes} [Mon Aug 22 19:05:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:05:52.654491 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d191aca-e8e6-41b9-a989-a60f52f1d9fa None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:52.688496 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515a0a54-4711-4b53-b339-9348dc7f14b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:52.689463 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515a0a54-4711-4b53-b339-9348dc7f14b6 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:52.689878 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515a0a54-4711-4b53-b339-9348dc7f14b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:52.690308 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515a0a54-4711-4b53-b339-9348dc7f14b6 None admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:52.691140 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515a0a54-4711-4b53-b339-9348dc7f14b6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:52.691560 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515a0a54-4711-4b53-b339-9348dc7f14b6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:52.693219 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515a0a54-4711-4b53-b339-9348dc7f14b6 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:52.714740 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 272/532] 10.222.0.61 () {56 vars in 1148 bytes} [Mon Aug 22 19:05:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:52.720838 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515a0a54-4711-4b53-b339-9348dc7f14b6 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:52.752331 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:52.774924 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d191aca-e8e6-41b9-a989-a60f52f1d9fa None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:52.783661 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:52.813881 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08fc939-b55e-4730-86aa-4c449930ea3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:52.814803 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08fc939-b55e-4730-86aa-4c449930ea3e None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:52.815322 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08fc939-b55e-4730-86aa-4c449930ea3e None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:52.815836 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08fc939-b55e-4730-86aa-4c449930ea3e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:52.824274 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515a0a54-4711-4b53-b339-9348dc7f14b6 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:52.858959 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548d663e-6cfd-444c-9479-3e66625d4278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:52.859702 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548d663e-6cfd-444c-9479-3e66625d4278 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:52.860058 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548d663e-6cfd-444c-9479-3e66625d4278 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:52.860401 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548d663e-6cfd-444c-9479-3e66625d4278 None admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:52.861187 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548d663e-6cfd-444c-9479-3e66625d4278 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:52.861567 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548d663e-6cfd-444c-9479-3e66625d4278 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:52.863079 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548d663e-6cfd-444c-9479-3e66625d4278 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:52.876256 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 273/533] 10.222.0.61 () {56 vars in 1148 bytes} [Mon Aug 22 19:05:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:52.877136 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548d663e-6cfd-444c-9479-3e66625d4278 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:52.891298 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08fc939-b55e-4730-86aa-4c449930ea3e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:52.893556 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08fc939-b55e-4730-86aa-4c449930ea3e None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:52.981000 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548d663e-6cfd-444c-9479-3e66625d4278 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:52.981812 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:53.006109 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 261/534] 10.222.0.61 () {58 vars in 1282 bytes} [Mon Aug 22 19:05:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 19:05:53.007201 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08fc939-b55e-4730-86aa-4c449930ea3e None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:53.011082 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:53.023915 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c01580-4449-4d6d-be72-fa06d43f46f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:53.024336 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c01580-4449-4d6d-be72-fa06d43f46f9 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:53.024595 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c01580-4449-4d6d-be72-fa06d43f46f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:53.024868 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c01580-4449-4d6d-be72-fa06d43f46f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:53.025602 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c01580-4449-4d6d-be72-fa06d43f46f9 None admin] RBAC: Adding query filter params (role.id=324bd04d55c34b61a5282c9f6ed73f46, scope.project.id=e07dc5ebdc504c6da706a2d21c0a3970, user.id=fe15d6a79ed44e5396d13cd29ba160ac) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:53.025910 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c01580-4449-4d6d-be72-fa06d43f46f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:53.027339 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c01580-4449-4d6d-be72-fa06d43f46f9 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:53.033214 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 274/535] 10.222.0.61 () {56 vars in 1374 bytes} [Mon Aug 22 19:05:52 2022] GET /identity/v3/role_assignments?user.id=fe15d6a79ed44e5396d13cd29ba160ac&scope.project.id=e07dc5ebdc504c6da706a2d21c0a3970&role.id=324bd04d55c34b61a5282c9f6ed73f46 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:53.034073 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c01580-4449-4d6d-be72-fa06d43f46f9 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:53.127508 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08fc939-b55e-4730-86aa-4c449930ea3e None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:53.152069 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c01580-4449-4d6d-be72-fa06d43f46f9 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:53.168043 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5509b890-5c6b-4da8-93c0-9fd54f13e4a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:53.169026 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5509b890-5c6b-4da8-93c0-9fd54f13e4a4 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:53.169513 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5509b890-5c6b-4da8-93c0-9fd54f13e4a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:53.170095 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5509b890-5c6b-4da8-93c0-9fd54f13e4a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:53.171123 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5509b890-5c6b-4da8-93c0-9fd54f13e4a4 None admin] RBAC: Adding query filter params (role.id=324bd04d55c34b61a5282c9f6ed73f46, scope.project.id=e07dc5ebdc504c6da706a2d21c0a3970, user.id=a4aca06c966a42f29f5c5bc108529d41) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:53.172209 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5509b890-5c6b-4da8-93c0-9fd54f13e4a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:53.173815 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5509b890-5c6b-4da8-93c0-9fd54f13e4a4 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:53.180539 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 262/536] 10.222.0.61 () {56 vars in 1374 bytes} [Mon Aug 22 19:05:53 2022] GET /identity/v3/role_assignments?user.id=a4aca06c966a42f29f5c5bc108529d41&scope.project.id=e07dc5ebdc504c6da706a2d21c0a3970&role.id=324bd04d55c34b61a5282c9f6ed73f46 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:53.189714 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf6c6d8-6c06-40a5-905c-61fe42973f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:53.190313 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf6c6d8-6c06-40a5-905c-61fe42973f09 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:53.190876 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf6c6d8-6c06-40a5-905c-61fe42973f09 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:53.191269 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf6c6d8-6c06-40a5-905c-61fe42973f09 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:53.204134 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf6c6d8-6c06-40a5-905c-61fe42973f09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:53.205823 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf6c6d8-6c06-40a5-905c-61fe42973f09 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:53.222202 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-cdf6c6d8-6c06-40a5-905c-61fe42973f09 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:05:53.222202 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:05:53.222202 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:05:53.222202 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:05:53.222202 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:05:53.222202 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:05:53.222202 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:05:53.222202 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:05:53.222202 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:05:53.222202 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:05:53.222202 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:05:53.222202 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:05:53.222202 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 19:05:53.222202 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 19:05:53.222202 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 19:05:53.222202 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 19:05:53.223200 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:05:53.223200 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:05:53.223200 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:05:53.223200 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:05:53.223200 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:05:53.223200 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:05:53.223200 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:05:53.223200 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:05:53.223200 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:05:53.223200 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:05:53.223200 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:05:53.223200 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:05:53.223200 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:05:53.223200 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:05:53.223200 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:05:53.223200 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:05:53.223200 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 19:05:53.224098 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 19:05:53.224098 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:05:53.224098 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:05:53.224098 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 275/537] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:05:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:05:53.233307 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5509b890-5c6b-4da8-93c0-9fd54f13e4a4 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:53.286280 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:53.356179 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5509b890-5c6b-4da8-93c0-9fd54f13e4a4 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:53.395365 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4d19f7-4f37-4b38-b799-b798e7683a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:53.396032 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4d19f7-4f37-4b38-b799-b798e7683a5a None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:53.396322 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4d19f7-4f37-4b38-b799-b798e7683a5a None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:53.396593 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4d19f7-4f37-4b38-b799-b798e7683a5a None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:53.397255 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4d19f7-4f37-4b38-b799-b798e7683a5a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:53.397560 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4d19f7-4f37-4b38-b799-b798e7683a5a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:53.398987 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4d19f7-4f37-4b38-b799-b798e7683a5a None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:53.413205 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 263/538] 10.222.0.61 () {56 vars in 1109 bytes} [Mon Aug 22 19:05:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:53.418514 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf6c6d8-6c06-40a5-905c-61fe42973f09 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:53.482217 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:53.554856 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf93546-5397-4749-9573-0731b5c13b7f None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:53.555153 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf93546-5397-4749-9573-0731b5c13b7f None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:53.555320 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf93546-5397-4749-9573-0731b5c13b7f None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:53.556939 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 264/539] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:05:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 19:05:53.560054 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf6c6d8-6c06-40a5-905c-61fe42973f09 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:53.572824 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a14c8b3-f6ac-4b36-855c-a7b3effbd1ad None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:53.573103 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a14c8b3-f6ac-4b36-855c-a7b3effbd1ad None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:53.573378 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a14c8b3-f6ac-4b36-855c-a7b3effbd1ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:53.597962 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82525e78-fc8e-4269-948c-8f82cf17b133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:53.598839 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82525e78-fc8e-4269-948c-8f82cf17b133 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:53.599363 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82525e78-fc8e-4269-948c-8f82cf17b133 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:53.599817 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82525e78-fc8e-4269-948c-8f82cf17b133 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:53.607004 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82525e78-fc8e-4269-948c-8f82cf17b133 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:53.608646 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-8a14c8b3-f6ac-4b36-855c-a7b3effbd1ad None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:05:53.608768 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82525e78-fc8e-4269-948c-8f82cf17b133 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:53.609884 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-8a14c8b3-f6ac-4b36-855c-a7b3effbd1ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:05:53.629183 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-82525e78-fc8e-4269-948c-8f82cf17b133 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 19:05:53.629183 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:05:53.629183 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:05:53.629183 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:05:53.629183 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:05:53.629183 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:05:53.629183 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:05:53.629183 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:05:53.629183 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:05:53.629183 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:05:53.629183 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:05:53.629183 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:05:53.629183 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 19:05:53.629183 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 19:05:53.629183 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 19:05:53.630090 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 19:05:53.630090 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:05:53.630090 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:05:53.630090 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:05:53.630090 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:05:53.630090 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:05:53.630090 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:05:53.630090 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:05:53.630090 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:05:53.630090 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:05:53.630090 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:05:53.630090 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:05:53.630090 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:05:53.630090 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:05:53.630090 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:05:53.630090 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:05:53.630090 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:05:53.635726 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 19:05:53.635726 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 19:05:53.635726 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 19:05:53.635726 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 19:05:53.635726 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 19:05:53.635726 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 19:05:53.635726 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 19:05:53.635726 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:05:53.635726 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 276/540] 10.222.0.61 () {56 vars in 1111 bytes} [Mon Aug 22 19:05:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:05:53.642265 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82525e78-fc8e-4269-948c-8f82cf17b133 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:53.708618 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 265/541] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:05:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 19:05:53.826916 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:53.831871 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82525e78-fc8e-4269-948c-8f82cf17b133 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:53.884497 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b781e27-5df0-478e-b98d-55d97c2f9eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:53.885115 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b781e27-5df0-478e-b98d-55d97c2f9eb1 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:53.885395 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b781e27-5df0-478e-b98d-55d97c2f9eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:53.885670 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b781e27-5df0-478e-b98d-55d97c2f9eb1 None admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:53.886365 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b781e27-5df0-478e-b98d-55d97c2f9eb1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:53.886672 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b781e27-5df0-478e-b98d-55d97c2f9eb1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:53.888299 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b781e27-5df0-478e-b98d-55d97c2f9eb1 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:53.906955 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 277/542] 10.222.0.61 () {56 vars in 1148 bytes} [Mon Aug 22 19:05:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 19:05:53.913718 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a14c8b3-f6ac-4b36-855c-a7b3effbd1ad None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:53.978745 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:54.073275 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a14c8b3-f6ac-4b36-855c-a7b3effbd1ad None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:54.119142 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6806a9c5-62cb-4888-b880-3707691621c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:54.119679 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6806a9c5-62cb-4888-b880-3707691621c4 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:54.119960 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6806a9c5-62cb-4888-b880-3707691621c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:54.120226 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6806a9c5-62cb-4888-b880-3707691621c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:54.120951 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6806a9c5-62cb-4888-b880-3707691621c4 None admin] RBAC: Adding query filter params (role.id=324bd04d55c34b61a5282c9f6ed73f46, scope.project.id=e07dc5ebdc504c6da706a2d21c0a3970, user.id=56e79498873d4ed29a0e19bbea9b9214) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:54.121260 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6806a9c5-62cb-4888-b880-3707691621c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:54.122667 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6806a9c5-62cb-4888-b880-3707691621c4 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:54.131751 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 266/543] 10.222.0.61 () {56 vars in 1374 bytes} [Mon Aug 22 19:05:53 2022] GET /identity/v3/role_assignments?user.id=56e79498873d4ed29a0e19bbea9b9214&scope.project.id=e07dc5ebdc504c6da706a2d21c0a3970&role.id=324bd04d55c34b61a5282c9f6ed73f46 => generated 259 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:54.151429 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b781e27-5df0-478e-b98d-55d97c2f9eb1 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:54.282632 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b781e27-5df0-478e-b98d-55d97c2f9eb1 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:54.320274 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd3cb04-7c65-4fb6-af66-8d9971031db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:54.321323 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd3cb04-7c65-4fb6-af66-8d9971031db2 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:54.321807 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd3cb04-7c65-4fb6-af66-8d9971031db2 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:54.322300 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd3cb04-7c65-4fb6-af66-8d9971031db2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:54.333826 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd3cb04-7c65-4fb6-af66-8d9971031db2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 19:05:54.339069 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-4fd3cb04-7c65-4fb6-af66-8d9971031db2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 19:05:54.339069 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:05:54.339069 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:05:54.339069 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:05:54.339069 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:05:54.339069 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:05:54.339069 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:05:54.339069 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:05:54.339069 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:05:54.339069 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:05:54.339069 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:05:54.339069 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:05:54.339069 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 19:05:54.339069 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 19:05:54.339069 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 19:05:54.339069 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 19:05:54.340008 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 19:05:54.340008 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 19:05:54.340008 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 19:05:54.340008 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 19:05:54.340008 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:05:54.340008 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:05:54.340008 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:05:54.340008 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:05:54.340008 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:05:54.340008 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:05:54.340008 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:05:54.340008 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:05:54.340008 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:05:54.340008 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:05:54.340008 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:05:54.340008 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:05:54.340818 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:05:54.340818 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:05:54.340818 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:05:54.340818 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:05:54.340818 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 19:05:54.340818 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 19:05:54.340818 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 19:05:54.340818 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 19:05:54.340818 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 19:05:54.340818 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 19:05:54.340818 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 19:05:54.340818 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:05:54.343432 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 278/544] 10.222.0.61 () {56 vars in 1102 bytes} [Mon Aug 22 19:05:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:05:54.351711 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6806a9c5-62cb-4888-b880-3707691621c4 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:54.485031 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b335c769-b13d-4b58-8a5b-5e4eb3d61bf1 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:54.485309 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b335c769-b13d-4b58-8a5b-5e4eb3d61bf1 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:54.485586 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b335c769-b13d-4b58-8a5b-5e4eb3d61bf1 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:54.486904 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 279/545] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:05:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 19:05:54.495678 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a83070c-3bc6-4a4a-b6c1-755a6aeb72d6 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:54.495678 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a83070c-3bc6-4a4a-b6c1-755a6aeb72d6 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:54.495678 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a83070c-3bc6-4a4a-b6c1-755a6aeb72d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:54.496004 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:54.501542 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6806a9c5-62cb-4888-b880-3707691621c4 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:54.516222 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-5a83070c-3bc6-4a4a-b6c1-755a6aeb72d6 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:05:54.518423 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-5a83070c-3bc6-4a4a-b6c1-755a6aeb72d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:05:54.548043 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:54.603753 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 280/546] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:05:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 19:05:54.616382 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6554c097-2316-4b59-afdc-ca0172de9f07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:54.617011 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6554c097-2316-4b59-afdc-ca0172de9f07 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:54.617289 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6554c097-2316-4b59-afdc-ca0172de9f07 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:54.617562 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6554c097-2316-4b59-afdc-ca0172de9f07 None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:54.618245 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6554c097-2316-4b59-afdc-ca0172de9f07 None admin] RBAC: Adding query filter params (name=service) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:54.618548 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6554c097-2316-4b59-afdc-ca0172de9f07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:54.620082 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6554c097-2316-4b59-afdc-ca0172de9f07 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:54.637979 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 267/547] 10.222.0.61 () {56 vars in 1103 bytes} [Mon Aug 22 19:05:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:54.643433 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a83070c-3bc6-4a4a-b6c1-755a6aeb72d6 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:54.778407 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a83070c-3bc6-4a4a-b6c1-755a6aeb72d6 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:54.813993 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1b29df-1f6d-44da-938a-82313a02e945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:54.814989 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1b29df-1f6d-44da-938a-82313a02e945 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:54.815483 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1b29df-1f6d-44da-938a-82313a02e945 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:54.815980 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1b29df-1f6d-44da-938a-82313a02e945 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:54.827217 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1b29df-1f6d-44da-938a-82313a02e945 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:54.828958 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1b29df-1f6d-44da-938a-82313a02e945 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:54.843290 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-be1b29df-1f6d-44da-938a-82313a02e945 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:05:54.843290 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:05:54.843290 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:05:54.843290 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:05:54.843290 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:05:54.843290 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:05:54.843290 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:05:54.843290 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:05:54.843290 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:05:54.843290 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:05:54.843290 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:05:54.843290 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:05:54.843290 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 19:05:54.843290 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 19:05:54.843290 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 19:05:54.843290 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 19:05:54.844264 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:05:54.844264 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:05:54.844264 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:05:54.844264 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:05:54.844264 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:05:54.844264 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:05:54.844264 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:05:54.844264 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:05:54.844264 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:05:54.844264 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:05:54.844264 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:05:54.844264 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:05:54.844264 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:05:54.844264 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:05:54.844264 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:05:54.844264 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:05:54.844264 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 19:05:54.845127 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 19:05:54.845127 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:05:54.845127 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:05:54.847223 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 281/548] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:05:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:05:54.852540 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6554c097-2316-4b59-afdc-ca0172de9f07 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:55.035031 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6554c097-2316-4b59-afdc-ca0172de9f07 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:55.035630 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1b29df-1f6d-44da-938a-82313a02e945 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:55.038431 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:55.096894 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f518e697-aba2-4346-b887-0c81d8496d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:55.097506 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f518e697-aba2-4346-b887-0c81d8496d15 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:55.097785 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f518e697-aba2-4346-b887-0c81d8496d15 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:55.098055 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f518e697-aba2-4346-b887-0c81d8496d15 None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:55.098706 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f518e697-aba2-4346-b887-0c81d8496d15 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:55.099005 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f518e697-aba2-4346-b887-0c81d8496d15 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:55.100398 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f518e697-aba2-4346-b887-0c81d8496d15 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:55.133872 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 268/549] 10.222.0.61 () {56 vars in 1109 bytes} [Mon Aug 22 19:05:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:55.138904 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f518e697-aba2-4346-b887-0c81d8496d15 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:55.158438 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1b29df-1f6d-44da-938a-82313a02e945 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:55.197245 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709199d7-62e7-4e92-b5a0-b391921602c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:55.198638 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709199d7-62e7-4e92-b5a0-b391921602c6 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:55.199117 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709199d7-62e7-4e92-b5a0-b391921602c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:55.199611 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709199d7-62e7-4e92-b5a0-b391921602c6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:55.199987 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:55.212729 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709199d7-62e7-4e92-b5a0-b391921602c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 19:05:55.219028 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-709199d7-62e7-4e92-b5a0-b391921602c6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 19:05:55.219028 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:05:55.219028 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:05:55.219028 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:05:55.219028 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:05:55.219028 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:05:55.219028 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:05:55.219028 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:05:55.219028 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:05:55.219028 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:05:55.219028 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:05:55.219028 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:05:55.219028 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 19:05:55.219028 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 19:05:55.219028 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 19:05:55.219028 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 19:05:55.219967 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 19:05:55.219967 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 19:05:55.219967 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 19:05:55.219967 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 19:05:55.219967 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:05:55.219967 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:05:55.219967 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:05:55.219967 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:05:55.219967 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:05:55.219967 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:05:55.219967 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:05:55.219967 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:05:55.219967 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:05:55.219967 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:05:55.219967 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:05:55.219967 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:05:55.221408 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:05:55.221408 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:05:55.221408 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:05:55.221408 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:05:55.221408 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 19:05:55.221408 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 19:05:55.221408 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 19:05:55.221408 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 19:05:55.221408 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 19:05:55.221408 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 19:05:55.221408 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 19:05:55.221408 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:05:55.229161 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 282/550] 10.222.0.61 () {56 vars in 1102 bytes} [Mon Aug 22 19:05:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 19:05:55.231018 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709199d7-62e7-4e92-b5a0-b391921602c6 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:55.303194 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:55.348993 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709199d7-62e7-4e92-b5a0-b391921602c6 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:55.398276 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f44ac93-fa2c-4ea6-bbc8-10698c7d85e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:55.399265 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f44ac93-fa2c-4ea6-bbc8-10698c7d85e8 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:55.399765 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f44ac93-fa2c-4ea6-bbc8-10698c7d85e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:55.400236 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f44ac93-fa2c-4ea6-bbc8-10698c7d85e8 None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:55.401103 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f44ac93-fa2c-4ea6-bbc8-10698c7d85e8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:55.401615 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f44ac93-fa2c-4ea6-bbc8-10698c7d85e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:55.403192 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f44ac93-fa2c-4ea6-bbc8-10698c7d85e8 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:55.420484 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 283/551] 10.222.0.61 () {56 vars in 1103 bytes} [Mon Aug 22 19:05:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 19:05:55.422350 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c38eb70-e244-486f-b941-d4067dd64d10 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:55.422629 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c38eb70-e244-486f-b941-d4067dd64d10 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:55.422899 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c38eb70-e244-486f-b941-d4067dd64d10 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:55.424398 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 284/552] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:05:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:05:55.429740 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbfb44e-2d85-483e-93be-e9c886ac348e None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:55.430000 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbfb44e-2d85-483e-93be-e9c886ac348e None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:55.430274 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbfb44e-2d85-483e-93be-e9c886ac348e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:55.436480 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f518e697-aba2-4346-b887-0c81d8496d15 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:55.456428 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-2dbfb44e-2d85-483e-93be-e9c886ac348e None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:05:55.457625 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-2dbfb44e-2d85-483e-93be-e9c886ac348e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:05:55.486134 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7ecd45-a809-4369-ab01-48bf237476e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:55.486776 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7ecd45-a809-4369-ab01-48bf237476e5 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:55.487054 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7ecd45-a809-4369-ab01-48bf237476e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:55.487324 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7ecd45-a809-4369-ab01-48bf237476e5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:55.495402 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7ecd45-a809-4369-ab01-48bf237476e5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:55.496889 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7ecd45-a809-4369-ab01-48bf237476e5 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:55.511366 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-4f7ecd45-a809-4369-ab01-48bf237476e5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 22 19:05:55.511366 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:05:55.511366 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:05:55.511366 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:05:55.511366 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:05:55.511366 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:05:55.511366 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:05:55.511366 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:05:55.511366 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:05:55.511366 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:05:55.511366 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:05:55.511366 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:05:55.511366 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 19:05:55.511366 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 19:05:55.511366 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 19:05:55.511366 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 19:05:55.512476 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:05:55.512476 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:05:55.512476 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 19:05:55.512476 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 19:05:55.512476 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 19:05:55.512476 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 19:05:55.512476 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:05:55.512476 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:05:55.512476 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:05:55.512476 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:05:55.512476 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:05:55.512476 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:05:55.512476 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:05:55.512476 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:05:55.512476 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:05:55.512476 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:05:55.512476 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:05:55.512476 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:05:55.513509 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:05:55.513509 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:05:55.513509 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 19:05:55.513509 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 19:05:55.513509 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 19:05:55.513509 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 19:05:55.513509 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 19:05:55.513509 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 19:05:55.513509 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 22 19:05:55.513509 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:05:55.513509 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 269/553] 10.222.0.61 () {56 vars in 1096 bytes} [Mon Aug 22 19:05:55 2022] GET /identity/v3/users/swift => generated 83 bytes in 374 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:05:55.513509 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7ecd45-a809-4369-ab01-48bf237476e5 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:55.552831 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 285/554] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:05:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 19:05:55.554885 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbfb44e-2d85-483e-93be-e9c886ac348e None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:55.606762 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:55.628590 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7ecd45-a809-4369-ab01-48bf237476e5 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:55.637662 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:55.667512 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c69058-5cc1-47b6-be9e-15babc9424e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:55.668584 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c69058-5cc1-47b6-be9e-15babc9424e7 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:55.669016 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c69058-5cc1-47b6-be9e-15babc9424e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:55.669416 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c69058-5cc1-47b6-be9e-15babc9424e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:55.681070 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c69058-5cc1-47b6-be9e-15babc9424e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:55.682727 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c69058-5cc1-47b6-be9e-15babc9424e7 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:55.696179 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbfb44e-2d85-483e-93be-e9c886ac348e None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:55.699179 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-69c69058-5cc1-47b6-be9e-15babc9424e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:05:55.699179 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:05:55.699179 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:05:55.699179 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:05:55.699179 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:05:55.699179 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:05:55.699179 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:05:55.699179 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:05:55.699179 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:05:55.699179 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:05:55.699179 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:05:55.699179 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:05:55.699179 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 19:05:55.699179 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 19:05:55.699179 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 19:05:55.699179 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 19:05:55.700260 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:05:55.700260 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:05:55.700260 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:05:55.700260 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:05:55.700260 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:05:55.700260 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:05:55.700260 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:05:55.700260 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:05:55.700260 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:05:55.700260 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:05:55.700260 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:05:55.700260 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:05:55.700260 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:05:55.700260 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:05:55.700260 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:05:55.700260 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:05:55.700260 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 19:05:55.701214 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 19:05:55.701214 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:05:55.701214 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:05:55.703738 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 270/555] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:05:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:05:55.704995 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447f2134-ad40-4ac0-a36a-4b9592f48e51 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:55.705286 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447f2134-ad40-4ac0-a36a-4b9592f48e51 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:55.705566 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447f2134-ad40-4ac0-a36a-4b9592f48e51 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:55.707101 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 271/556] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:05:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:05:55.709923 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447f2134-ad40-4ac0-a36a-4b9592f48e51 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:55.741383 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b5c6a9-94e8-4794-91d8-f66f12815c9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:55.742143 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b5c6a9-94e8-4794-91d8-f66f12815c9f None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:55.742332 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b5c6a9-94e8-4794-91d8-f66f12815c9f None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:55.742608 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b5c6a9-94e8-4794-91d8-f66f12815c9f None admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:55.743378 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b5c6a9-94e8-4794-91d8-f66f12815c9f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:55.743659 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b5c6a9-94e8-4794-91d8-f66f12815c9f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:55.745144 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b5c6a9-94e8-4794-91d8-f66f12815c9f None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:55.763963 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 286/557] 10.222.0.61 () {56 vars in 1135 bytes} [Mon Aug 22 19:05:55 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:55.765270 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caefb55-3395-4523-a78d-d907a5e091b4 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:55.766071 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caefb55-3395-4523-a78d-d907a5e091b4 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:55.766347 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caefb55-3395-4523-a78d-d907a5e091b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:55.784537 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-9caefb55-3395-4523-a78d-d907a5e091b4 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:05:55.785786 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-9caefb55-3395-4523-a78d-d907a5e091b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:05:55.832749 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447f2134-ad40-4ac0-a36a-4b9592f48e51 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:55.859804 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 287/558] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:05:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 19:05:55.860446 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caefb55-3395-4523-a78d-d907a5e091b4 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:55.867067 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435bb890-98b9-4bad-a0dc-051df5eabc21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:55.867896 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435bb890-98b9-4bad-a0dc-051df5eabc21 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:55.868570 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:55.869017 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435bb890-98b9-4bad-a0dc-051df5eabc21 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:55.869465 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435bb890-98b9-4bad-a0dc-051df5eabc21 None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:55.870274 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435bb890-98b9-4bad-a0dc-051df5eabc21 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:55.870758 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435bb890-98b9-4bad-a0dc-051df5eabc21 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:55.872495 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435bb890-98b9-4bad-a0dc-051df5eabc21 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:55.886445 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 272/559] 10.222.0.61 () {56 vars in 1109 bytes} [Mon Aug 22 19:05:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:55.887539 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435bb890-98b9-4bad-a0dc-051df5eabc21 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:55.931056 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:56.002887 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435bb890-98b9-4bad-a0dc-051df5eabc21 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:56.008572 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caefb55-3395-4523-a78d-d907a5e091b4 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:56.046193 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b7cc2a-ad40-4efd-a21a-6ec025b25dbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:56.047209 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b7cc2a-ad40-4efd-a21a-6ec025b25dbf None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:56.047745 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b7cc2a-ad40-4efd-a21a-6ec025b25dbf None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:56.048193 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b7cc2a-ad40-4efd-a21a-6ec025b25dbf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:56.049846 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392074ca-eb9c-46af-a0fa-16fae613adb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:56.050482 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392074ca-eb9c-46af-a0fa-16fae613adb3 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:56.050962 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392074ca-eb9c-46af-a0fa-16fae613adb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:56.051112 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392074ca-eb9c-46af-a0fa-16fae613adb3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:56.064753 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392074ca-eb9c-46af-a0fa-16fae613adb3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 19:05:56.070884 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-392074ca-eb9c-46af-a0fa-16fae613adb3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 19:05:56.070884 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:05:56.070884 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:05:56.070884 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:05:56.070884 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:05:56.070884 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:05:56.070884 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:05:56.070884 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:05:56.070884 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:05:56.070884 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:05:56.070884 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:05:56.070884 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:05:56.070884 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 19:05:56.070884 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 19:05:56.070884 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 19:05:56.070884 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 19:05:56.071876 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 19:05:56.071876 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 19:05:56.071876 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 19:05:56.071876 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 19:05:56.071876 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:05:56.071876 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:05:56.071876 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:05:56.071876 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:05:56.071876 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:05:56.071876 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:05:56.071876 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:05:56.071876 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:05:56.071876 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:05:56.071876 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:05:56.071876 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:05:56.071876 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:05:56.072733 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:05:56.072733 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:05:56.072733 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:05:56.072733 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:05:56.072733 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 19:05:56.072733 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 19:05:56.072733 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 19:05:56.072733 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 19:05:56.072733 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 19:05:56.072733 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 19:05:56.072733 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 19:05:56.072733 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:05:56.072733 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 273/560] 10.222.0.61 () {56 vars in 1102 bytes} [Mon Aug 22 19:05:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:05:56.073428 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392074ca-eb9c-46af-a0fa-16fae613adb3 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:56.131070 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b7cc2a-ad40-4efd-a21a-6ec025b25dbf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:56.133025 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b7cc2a-ad40-4efd-a21a-6ec025b25dbf None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:56.151022 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:56.180730 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:56.203608 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392074ca-eb9c-46af-a0fa-16fae613adb3 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:56.241022 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164e6f8c-890c-4f72-99bd-1c2615ded70e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:56.241640 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164e6f8c-890c-4f72-99bd-1c2615ded70e None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:56.241914 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164e6f8c-890c-4f72-99bd-1c2615ded70e None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:56.246788 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164e6f8c-890c-4f72-99bd-1c2615ded70e None admin] PATH_INFO: `/v3/users/nova` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:56.246921 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 288/561] 10.222.0.61 () {58 vars in 1282 bytes} [Mon Aug 22 19:05:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 19:05:56.246921 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b7cc2a-ad40-4efd-a21a-6ec025b25dbf None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:56.254744 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164e6f8c-890c-4f72-99bd-1c2615ded70e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:56.256239 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164e6f8c-890c-4f72-99bd-1c2615ded70e None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:56.267861 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-164e6f8c-890c-4f72-99bd-1c2615ded70e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 22 19:05:56.267861 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:05:56.267861 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:05:56.267861 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:05:56.267861 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:05:56.267861 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:05:56.267861 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:05:56.267861 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:05:56.267861 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:05:56.267861 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:05:56.267861 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:05:56.267861 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:05:56.267861 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 19:05:56.267861 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 19:05:56.267861 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 19:05:56.267861 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 19:05:56.268841 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:05:56.268841 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:05:56.268841 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 19:05:56.268841 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 19:05:56.268841 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 19:05:56.268841 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 19:05:56.268841 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:05:56.268841 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:05:56.268841 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:05:56.268841 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:05:56.268841 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:05:56.268841 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:05:56.268841 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:05:56.268841 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:05:56.268841 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:05:56.268841 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:05:56.268841 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:05:56.268841 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:05:56.269829 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:05:56.269829 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:05:56.269829 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 19:05:56.269829 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 19:05:56.269829 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 19:05:56.269829 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 19:05:56.269829 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 19:05:56.269829 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 19:05:56.269829 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 22 19:05:56.269829 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:05:56.269829 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 274/562] 10.222.0.61 () {56 vars in 1093 bytes} [Mon Aug 22 19:05:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:05:56.269829 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164e6f8c-890c-4f72-99bd-1c2615ded70e None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:56.352937 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b7cc2a-ad40-4efd-a21a-6ec025b25dbf None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:56.375403 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164e6f8c-890c-4f72-99bd-1c2615ded70e None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:56.388697 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d8bfb8-69a7-4035-9471-76d31894b6fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:56.389241 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d8bfb8-69a7-4035-9471-76d31894b6fc None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:56.389375 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:56.389513 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d8bfb8-69a7-4035-9471-76d31894b6fc None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:56.389777 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d8bfb8-69a7-4035-9471-76d31894b6fc None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:56.390466 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d8bfb8-69a7-4035-9471-76d31894b6fc None admin] RBAC: Adding query filter params (name=service) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:56.390831 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d8bfb8-69a7-4035-9471-76d31894b6fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:56.392166 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d8bfb8-69a7-4035-9471-76d31894b6fc None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:56.404555 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 289/563] 10.222.0.61 () {56 vars in 1103 bytes} [Mon Aug 22 19:05:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:56.405451 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d8bfb8-69a7-4035-9471-76d31894b6fc None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:56.408855 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:56.414001 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db21fa1-678d-4c82-8def-68911b2f7f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:56.414899 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db21fa1-678d-4c82-8def-68911b2f7f93 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:56.415334 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db21fa1-678d-4c82-8def-68911b2f7f93 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:56.415775 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db21fa1-678d-4c82-8def-68911b2f7f93 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:56.426822 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db21fa1-678d-4c82-8def-68911b2f7f93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 19:05:56.431957 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-9db21fa1-678d-4c82-8def-68911b2f7f93 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 19:05:56.431957 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:05:56.431957 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:05:56.431957 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:05:56.431957 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:05:56.431957 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:05:56.431957 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:05:56.431957 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:05:56.431957 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:05:56.431957 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:05:56.431957 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:05:56.431957 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:05:56.431957 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 19:05:56.431957 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 19:05:56.431957 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 19:05:56.431957 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 19:05:56.432917 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 19:05:56.432917 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 19:05:56.432917 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 19:05:56.432917 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 19:05:56.432917 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:05:56.432917 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:05:56.432917 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:05:56.432917 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:05:56.432917 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:05:56.432917 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:05:56.432917 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:05:56.432917 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:05:56.432917 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:05:56.432917 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:05:56.432917 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:05:56.432917 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:05:56.433766 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:05:56.433766 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:05:56.433766 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:05:56.433766 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:05:56.433766 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 19:05:56.433766 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 19:05:56.433766 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 19:05:56.433766 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 19:05:56.433766 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 19:05:56.433766 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 19:05:56.433766 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 19:05:56.433766 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:05:56.433766 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 275/564] 10.222.0.61 () {56 vars in 1102 bytes} [Mon Aug 22 19:05:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:05:56.434461 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db21fa1-678d-4c82-8def-68911b2f7f93 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:56.509723 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d8bfb8-69a7-4035-9471-76d31894b6fc None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:56.532650 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db21fa1-678d-4c82-8def-68911b2f7f93 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:56.542566 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83e325d-171f-49f8-83c2-899826b7c181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:56.543188 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83e325d-171f-49f8-83c2-899826b7c181 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:56.543468 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83e325d-171f-49f8-83c2-899826b7c181 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:56.543758 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83e325d-171f-49f8-83c2-899826b7c181 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:56.554502 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83e325d-171f-49f8-83c2-899826b7c181 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:56.555952 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83e325d-171f-49f8-83c2-899826b7c181 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:56.565807 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8064b2c-cd22-4192-abbe-00188c4db553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:56.566407 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8064b2c-cd22-4192-abbe-00188c4db553 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:56.566689 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8064b2c-cd22-4192-abbe-00188c4db553 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:56.566962 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8064b2c-cd22-4192-abbe-00188c4db553 None admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:56.567718 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8064b2c-cd22-4192-abbe-00188c4db553 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:56.568029 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8064b2c-cd22-4192-abbe-00188c4db553 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:56.569417 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-a83e325d-171f-49f8-83c2-899826b7c181 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:05:56.569417 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:05:56.569417 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:05:56.569417 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:05:56.569417 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:05:56.569417 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:05:56.569417 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:05:56.569417 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:05:56.569417 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:05:56.569417 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:05:56.569417 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:05:56.569417 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:05:56.569417 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 19:05:56.569417 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 19:05:56.569417 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 19:05:56.569417 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 19:05:56.570326 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:05:56.570326 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:05:56.570326 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:05:56.570326 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:05:56.570326 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:05:56.570326 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:05:56.570326 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:05:56.570326 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:05:56.570326 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:05:56.570326 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:05:56.570326 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:05:56.570326 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:05:56.570326 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:05:56.570326 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:05:56.570326 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:05:56.570326 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:05:56.570326 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 19:05:56.571221 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 19:05:56.571221 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:05:56.571221 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:05:56.571414 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8064b2c-cd22-4192-abbe-00188c4db553 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:56.571484 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 290/565] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:05:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:05:56.571484 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83e325d-171f-49f8-83c2-899826b7c181 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:56.587206 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 276/566] 10.222.0.61 () {56 vars in 1133 bytes} [Mon Aug 22 19:05:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:56.588556 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd163d3-4789-45cd-b854-af4ae1f686be None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:56.588846 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd163d3-4789-45cd-b854-af4ae1f686be None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:56.589122 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd163d3-4789-45cd-b854-af4ae1f686be None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:56.590317 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 277/567] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:05:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:05:56.594950 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd163d3-4789-45cd-b854-af4ae1f686be None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:56.621576 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:56.633608 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:56.677525 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83e325d-171f-49f8-83c2-899826b7c181 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:56.693486 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd163d3-4789-45cd-b854-af4ae1f686be None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:56.709362 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05315d61-08e3-4721-93be-815cb57f5969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:56.709984 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05315d61-08e3-4721-93be-815cb57f5969 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:56.710259 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05315d61-08e3-4721-93be-815cb57f5969 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:56.710525 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05315d61-08e3-4721-93be-815cb57f5969 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:56.721085 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05315d61-08e3-4721-93be-815cb57f5969 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:56.722546 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05315d61-08e3-4721-93be-815cb57f5969 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:56.726301 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04e61f2-dcaa-4eb4-a8d6-e21d9363fb1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:56.726892 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04e61f2-dcaa-4eb4-a8d6-e21d9363fb1d None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:56.727169 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04e61f2-dcaa-4eb4-a8d6-e21d9363fb1d None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:56.727441 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04e61f2-dcaa-4eb4-a8d6-e21d9363fb1d None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:56.728211 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04e61f2-dcaa-4eb4-a8d6-e21d9363fb1d None admin] RBAC: Adding query filter params (name=service) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:56.728514 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04e61f2-dcaa-4eb4-a8d6-e21d9363fb1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:56.729853 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04e61f2-dcaa-4eb4-a8d6-e21d9363fb1d None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:56.736709 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-05315d61-08e3-4721-93be-815cb57f5969 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:05:56.736709 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:05:56.736709 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:05:56.736709 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:05:56.736709 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:05:56.736709 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:05:56.736709 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:05:56.736709 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:05:56.736709 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:05:56.736709 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:05:56.736709 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:05:56.736709 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:05:56.736709 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 19:05:56.736709 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 19:05:56.736709 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 19:05:56.736709 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 19:05:56.737824 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:05:56.737824 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:05:56.737824 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:05:56.737824 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:05:56.737824 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:05:56.737824 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:05:56.737824 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:05:56.737824 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:05:56.737824 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:05:56.737824 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:05:56.737824 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:05:56.737824 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:05:56.737824 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:05:56.737824 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:05:56.737824 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:05:56.737824 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:05:56.737824 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 19:05:56.738734 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 19:05:56.738734 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:05:56.738734 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:05:56.738734 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 291/568] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:05:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:05:56.739217 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 278/569] 10.222.0.61 () {56 vars in 1103 bytes} [Mon Aug 22 19:05:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:56.739299 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05315d61-08e3-4721-93be-815cb57f5969 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:56.739370 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04e61f2-dcaa-4eb4-a8d6-e21d9363fb1d None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:56.828738 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:56.837915 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04e61f2-dcaa-4eb4-a8d6-e21d9363fb1d None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:56.841145 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05315d61-08e3-4721-93be-815cb57f5969 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:56.850664 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:56.872313 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a06399-9a29-4105-8a51-f479fd2e3ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:56.872944 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a06399-9a29-4105-8a51-f479fd2e3ed0 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:56.873221 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a06399-9a29-4105-8a51-f479fd2e3ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:56.873489 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a06399-9a29-4105-8a51-f479fd2e3ed0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:56.876010 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fe4b86-f953-45c7-b2b8-ba5f65abeaa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:56.876609 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fe4b86-f953-45c7-b2b8-ba5f65abeaa1 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:56.876887 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fe4b86-f953-45c7-b2b8-ba5f65abeaa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:56.877160 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fe4b86-f953-45c7-b2b8-ba5f65abeaa1 None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:56.877801 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fe4b86-f953-45c7-b2b8-ba5f65abeaa1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:56.878105 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fe4b86-f953-45c7-b2b8-ba5f65abeaa1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:56.879468 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fe4b86-f953-45c7-b2b8-ba5f65abeaa1 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:56.884273 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a06399-9a29-4105-8a51-f479fd2e3ed0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:56.885682 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a06399-9a29-4105-8a51-f479fd2e3ed0 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:56.892505 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 292/570] 10.222.0.61 () {56 vars in 1109 bytes} [Mon Aug 22 19:05:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:56.893661 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa2a85c-024b-46fc-ad91-bcdcd977b0bf None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:56.893945 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa2a85c-024b-46fc-ad91-bcdcd977b0bf None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:56.894217 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa2a85c-024b-46fc-ad91-bcdcd977b0bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:56.903365 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-83a06399-9a29-4105-8a51-f479fd2e3ed0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:05:56.903365 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:05:56.903365 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:05:56.903365 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:05:56.903365 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:05:56.903365 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:05:56.903365 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:05:56.903365 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:05:56.903365 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:05:56.903365 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:05:56.903365 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:05:56.903365 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:05:56.903365 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 19:05:56.903365 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 19:05:56.903365 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 19:05:56.903365 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 19:05:56.904382 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:05:56.904382 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:05:56.904382 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:05:56.904382 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:05:56.904382 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:05:56.904382 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:05:56.904382 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:05:56.904382 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:05:56.904382 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:05:56.904382 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:05:56.904382 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:05:56.904382 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:05:56.904382 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:05:56.904382 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:05:56.904382 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:05:56.904382 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:05:56.904382 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 19:05:56.905272 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 19:05:56.905272 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:05:56.905272 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:05:56.905272 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 279/571] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:05:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:05:56.906541 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a06399-9a29-4105-8a51-f479fd2e3ed0 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:56.915270 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-2aa2a85c-024b-46fc-ad91-bcdcd977b0bf None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:05:56.916498 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-2aa2a85c-024b-46fc-ad91-bcdcd977b0bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:05:56.982395 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 293/572] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:05:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 19:05:56.983229 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa2a85c-024b-46fc-ad91-bcdcd977b0bf None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:57.026965 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a06399-9a29-4105-8a51-f479fd2e3ed0 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:57.064583 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d770f0d-b353-422e-bdab-bb7d189d07be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:57.065235 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d770f0d-b353-422e-bdab-bb7d189d07be None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:57.065452 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d770f0d-b353-422e-bdab-bb7d189d07be None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:57.065724 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d770f0d-b353-422e-bdab-bb7d189d07be None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:57.066392 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d770f0d-b353-422e-bdab-bb7d189d07be None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:57.066694 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d770f0d-b353-422e-bdab-bb7d189d07be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:57.068105 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d770f0d-b353-422e-bdab-bb7d189d07be None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:57.084630 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 280/573] 10.222.0.61 () {56 vars in 1109 bytes} [Mon Aug 22 19:05:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:57.084630 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d770f0d-b353-422e-bdab-bb7d189d07be None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:57.087310 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:57.089593 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:57.094201 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa2a85c-024b-46fc-ad91-bcdcd977b0bf None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:57.133302 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0702ec1b-d3eb-4fe5-9492-445b542bb755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:57.133955 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0702ec1b-d3eb-4fe5-9492-445b542bb755 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:57.134235 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0702ec1b-d3eb-4fe5-9492-445b542bb755 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:57.134515 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0702ec1b-d3eb-4fe5-9492-445b542bb755 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:57.148858 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0702ec1b-d3eb-4fe5-9492-445b542bb755 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:57.150320 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0702ec1b-d3eb-4fe5-9492-445b542bb755 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:57.165510 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-0702ec1b-d3eb-4fe5-9492-445b542bb755 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:05:57.165510 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:05:57.165510 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:05:57.165510 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:05:57.165510 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:05:57.165510 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:05:57.165510 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:05:57.165510 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:05:57.165510 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:05:57.165510 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:05:57.165510 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:05:57.165510 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:05:57.165510 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 19:05:57.165510 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 19:05:57.165510 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 19:05:57.165510 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 19:05:57.166458 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:05:57.166458 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:05:57.166458 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:05:57.166458 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:05:57.166458 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:05:57.166458 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:05:57.166458 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:05:57.166458 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:05:57.166458 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:05:57.166458 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:05:57.166458 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:05:57.166458 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:05:57.166458 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:05:57.166458 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:05:57.166458 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:05:57.166458 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:05:57.166458 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 19:05:57.167353 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 19:05:57.167353 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:05:57.167353 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:05:57.167353 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 294/574] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:05:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:05:57.167353 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0702ec1b-d3eb-4fe5-9492-445b542bb755 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:57.195261 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d770f0d-b353-422e-bdab-bb7d189d07be None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:57.232570 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc6639c-39f9-43e4-92c7-583de0865130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:57.233135 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc6639c-39f9-43e4-92c7-583de0865130 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:57.233412 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc6639c-39f9-43e4-92c7-583de0865130 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:57.233688 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc6639c-39f9-43e4-92c7-583de0865130 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:57.240518 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc6639c-39f9-43e4-92c7-583de0865130 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:57.242001 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc6639c-39f9-43e4-92c7-583de0865130 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:57.253644 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-7dc6639c-39f9-43e4-92c7-583de0865130 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 19:05:57.253644 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:05:57.253644 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:05:57.253644 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:05:57.253644 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:05:57.253644 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:05:57.253644 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:05:57.253644 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:05:57.253644 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:05:57.253644 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:05:57.253644 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:05:57.253644 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:05:57.253644 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 19:05:57.253644 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 19:05:57.253644 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 19:05:57.254675 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 19:05:57.254675 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:05:57.254675 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:05:57.254675 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:05:57.254675 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:05:57.254675 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:05:57.254675 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:05:57.254675 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:05:57.254675 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:05:57.254675 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:05:57.254675 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:05:57.254675 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:05:57.254675 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:05:57.254675 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:05:57.254675 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:05:57.254675 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:05:57.254675 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:05:57.255571 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 19:05:57.255571 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 19:05:57.255571 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 19:05:57.255571 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 19:05:57.255571 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 19:05:57.255571 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 19:05:57.255571 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 19:05:57.255571 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:05:57.255571 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 281/575] 10.222.0.61 () {56 vars in 1111 bytes} [Mon Aug 22 19:05:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:05:57.256106 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc6639c-39f9-43e4-92c7-583de0865130 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:57.281422 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0702ec1b-d3eb-4fe5-9492-445b542bb755 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:57.314977 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3520bdc-79c2-4307-afc9-9b8739338645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:57.315573 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3520bdc-79c2-4307-afc9-9b8739338645 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:57.315874 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3520bdc-79c2-4307-afc9-9b8739338645 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:57.316143 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3520bdc-79c2-4307-afc9-9b8739338645 None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:57.316773 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3520bdc-79c2-4307-afc9-9b8739338645 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:57.317071 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3520bdc-79c2-4307-afc9-9b8739338645 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:57.318425 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3520bdc-79c2-4307-afc9-9b8739338645 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:57.318658 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:57.330780 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 295/576] 10.222.0.61 () {56 vars in 1109 bytes} [Mon Aug 22 19:05:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:57.331569 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3520bdc-79c2-4307-afc9-9b8739338645 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:57.343391 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:57.360125 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc6639c-39f9-43e4-92c7-583de0865130 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:57.393619 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e05bf3-94fc-48d9-a069-892564b7ff1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:57.394239 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e05bf3-94fc-48d9-a069-892564b7ff1c None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:57.394516 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e05bf3-94fc-48d9-a069-892564b7ff1c None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:57.394788 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e05bf3-94fc-48d9-a069-892564b7ff1c None admin] PATH_INFO: `/v3/users/glance` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:57.402090 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e05bf3-94fc-48d9-a069-892564b7ff1c None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:57.403520 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e05bf3-94fc-48d9-a069-892564b7ff1c None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:57.414315 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-d9e05bf3-94fc-48d9-a069-892564b7ff1c None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 22 19:05:57.414315 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:05:57.414315 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:05:57.414315 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:05:57.414315 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:05:57.414315 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:05:57.414315 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:05:57.414315 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:05:57.414315 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:05:57.414315 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:05:57.414315 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:05:57.414315 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:05:57.414315 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 19:05:57.414315 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 19:05:57.414315 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 19:05:57.414315 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 19:05:57.415351 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:05:57.415351 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:05:57.415351 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 19:05:57.415351 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 19:05:57.415351 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 19:05:57.415351 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 19:05:57.415351 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:05:57.415351 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:05:57.415351 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:05:57.415351 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:05:57.415351 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:05:57.415351 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:05:57.415351 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:05:57.415351 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:05:57.415351 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:05:57.415351 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:05:57.415351 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:05:57.415351 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:05:57.416493 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:05:57.416493 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:05:57.416493 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 19:05:57.416493 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 19:05:57.416493 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 19:05:57.416493 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 19:05:57.416493 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 19:05:57.416493 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 19:05:57.416493 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 22 19:05:57.416493 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:05:57.416493 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 282/577] 10.222.0.61 () {56 vars in 1099 bytes} [Mon Aug 22 19:05:57 2022] GET /identity/v3/users/glance => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:05:57.416493 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e05bf3-94fc-48d9-a069-892564b7ff1c None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:57.432976 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3520bdc-79c2-4307-afc9-9b8739338645 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:57.467567 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8f41bf-a608-44eb-ac3e-3b06149c969d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:57.468194 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8f41bf-a608-44eb-ac3e-3b06149c969d None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:57.468495 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8f41bf-a608-44eb-ac3e-3b06149c969d None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:57.468770 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8f41bf-a608-44eb-ac3e-3b06149c969d None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:57.469400 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8f41bf-a608-44eb-ac3e-3b06149c969d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:57.469697 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8f41bf-a608-44eb-ac3e-3b06149c969d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:57.471059 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8f41bf-a608-44eb-ac3e-3b06149c969d None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:57.481974 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 296/578] 10.222.0.61 () {56 vars in 1109 bytes} [Mon Aug 22 19:05:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:57.482724 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8f41bf-a608-44eb-ac3e-3b06149c969d None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:57.516494 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e05bf3-94fc-48d9-a069-892564b7ff1c None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:57.538737 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:57.549761 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d18e00-f07e-43ae-8793-8f5d0aca8409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:57.550389 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d18e00-f07e-43ae-8793-8f5d0aca8409 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:57.550675 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d18e00-f07e-43ae-8793-8f5d0aca8409 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:57.550952 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d18e00-f07e-43ae-8793-8f5d0aca8409 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:57.552226 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:57.561276 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d18e00-f07e-43ae-8793-8f5d0aca8409 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 19:05:57.566944 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-c6d18e00-f07e-43ae-8793-8f5d0aca8409 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 19:05:57.566944 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:05:57.566944 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:05:57.566944 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:05:57.566944 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:05:57.566944 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:05:57.566944 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:05:57.566944 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:05:57.566944 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:05:57.566944 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:05:57.566944 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:05:57.566944 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:05:57.566944 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 19:05:57.566944 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 19:05:57.566944 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 19:05:57.566944 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 19:05:57.567970 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 19:05:57.567970 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 19:05:57.567970 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 19:05:57.567970 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 19:05:57.567970 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:05:57.567970 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:05:57.567970 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:05:57.567970 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:05:57.567970 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:05:57.567970 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:05:57.567970 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:05:57.567970 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:05:57.567970 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:05:57.567970 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:05:57.567970 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:05:57.567970 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:05:57.568852 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:05:57.568852 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:05:57.568852 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:05:57.568852 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:05:57.568852 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 19:05:57.568852 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 19:05:57.568852 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 19:05:57.568852 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 19:05:57.568852 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 19:05:57.568852 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 19:05:57.568852 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 19:05:57.568852 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:05:57.568852 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 283/579] 10.222.0.61 () {56 vars in 1102 bytes} [Mon Aug 22 19:05:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:05:57.569539 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d18e00-f07e-43ae-8793-8f5d0aca8409 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:57.586436 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8f41bf-a608-44eb-ac3e-3b06149c969d None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:57.620653 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61db173a-8ed0-4d73-87b4-9160e55649c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:57.621248 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61db173a-8ed0-4d73-87b4-9160e55649c1 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:57.621520 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61db173a-8ed0-4d73-87b4-9160e55649c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:57.621789 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61db173a-8ed0-4d73-87b4-9160e55649c1 None admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:57.622468 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61db173a-8ed0-4d73-87b4-9160e55649c1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:57.622766 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61db173a-8ed0-4d73-87b4-9160e55649c1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:57.624226 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61db173a-8ed0-4d73-87b4-9160e55649c1 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:57.641268 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 297/580] 10.222.0.61 () {56 vars in 1148 bytes} [Mon Aug 22 19:05:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:57.641268 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61db173a-8ed0-4d73-87b4-9160e55649c1 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:57.670666 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d18e00-f07e-43ae-8793-8f5d0aca8409 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:57.703138 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2978b6-66d5-4233-91a0-b024d3a14682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:57.703785 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2978b6-66d5-4233-91a0-b024d3a14682 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:57.704063 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2978b6-66d5-4233-91a0-b024d3a14682 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:57.704330 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2978b6-66d5-4233-91a0-b024d3a14682 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:57.710744 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2978b6-66d5-4233-91a0-b024d3a14682 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:57.712207 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2978b6-66d5-4233-91a0-b024d3a14682 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:57.723061 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-6a2978b6-66d5-4233-91a0-b024d3a14682 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 19:05:57.723061 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:05:57.723061 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:05:57.723061 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:05:57.723061 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:05:57.723061 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:05:57.723061 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:05:57.723061 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:05:57.723061 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:05:57.723061 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:05:57.723061 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:05:57.723061 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:05:57.723061 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 19:05:57.723061 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 19:05:57.723061 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 19:05:57.724072 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 19:05:57.724072 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:05:57.724072 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:05:57.724072 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:05:57.724072 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:05:57.724072 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:05:57.724072 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:05:57.724072 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:05:57.724072 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:05:57.724072 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:05:57.724072 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:05:57.724072 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:05:57.724072 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:05:57.724072 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:05:57.724072 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:05:57.724072 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:05:57.724072 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:05:57.724991 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 19:05:57.724991 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 19:05:57.724991 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 19:05:57.724991 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 19:05:57.724991 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 19:05:57.724991 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 19:05:57.724991 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 19:05:57.724991 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:05:57.724991 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 284/581] 10.222.0.61 () {56 vars in 1111 bytes} [Mon Aug 22 19:05:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:05:57.724991 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2978b6-66d5-4233-91a0-b024d3a14682 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:57.738052 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61db173a-8ed0-4d73-87b4-9160e55649c1 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:57.766264 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:57.770438 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:57.773039 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528a475d-73bf-47dd-9c56-9cf74628fd9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:57.773720 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528a475d-73bf-47dd-9c56-9cf74628fd9a None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:57.773991 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528a475d-73bf-47dd-9c56-9cf74628fd9a None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:57.774257 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528a475d-73bf-47dd-9c56-9cf74628fd9a None admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:57.774982 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528a475d-73bf-47dd-9c56-9cf74628fd9a None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:57.775278 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528a475d-73bf-47dd-9c56-9cf74628fd9a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:57.776730 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528a475d-73bf-47dd-9c56-9cf74628fd9a None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:57.793678 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 298/582] 10.222.0.61 () {56 vars in 1137 bytes} [Mon Aug 22 19:05:57 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:57.794634 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528a475d-73bf-47dd-9c56-9cf74628fd9a None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:57.825894 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2978b6-66d5-4233-91a0-b024d3a14682 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:57.858306 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133b788c-642b-4615-acc2-1dea65f9f59c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:57.858930 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133b788c-642b-4615-acc2-1dea65f9f59c None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:57.859768 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133b788c-642b-4615-acc2-1dea65f9f59c None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:57.860043 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133b788c-642b-4615-acc2-1dea65f9f59c None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:57.867542 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133b788c-642b-4615-acc2-1dea65f9f59c None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:57.868992 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133b788c-642b-4615-acc2-1dea65f9f59c None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:57.880479 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-133b788c-642b-4615-acc2-1dea65f9f59c None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 22 19:05:57.880479 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:05:57.880479 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:05:57.880479 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:05:57.880479 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:05:57.880479 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:05:57.880479 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:05:57.880479 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:05:57.880479 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:05:57.880479 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:05:57.880479 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:05:57.880479 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:05:57.880479 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 19:05:57.880479 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 19:05:57.880479 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 19:05:57.880479 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 19:05:57.881609 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:05:57.881609 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:05:57.881609 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 19:05:57.881609 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 19:05:57.881609 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 19:05:57.881609 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 19:05:57.881609 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:05:57.881609 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:05:57.881609 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:05:57.881609 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:05:57.881609 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:05:57.881609 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:05:57.881609 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:05:57.881609 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:05:57.881609 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:05:57.881609 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:05:57.881609 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:05:57.881609 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:05:57.883219 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:05:57.883219 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:05:57.883219 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 19:05:57.883219 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 19:05:57.883219 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 19:05:57.883219 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 19:05:57.883219 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 19:05:57.883219 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 19:05:57.883219 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 22 19:05:57.883219 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:05:57.886263 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 285/583] 10.222.0.61 () {56 vars in 1102 bytes} [Mon Aug 22 19:05:57 2022] GET /identity/v3/users/neutron => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:05:57.886969 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133b788c-642b-4615-acc2-1dea65f9f59c None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:57.894397 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528a475d-73bf-47dd-9c56-9cf74628fd9a None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:57.928480 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8461f276-a695-4611-a4a6-58dddcb808ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:57.929078 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8461f276-a695-4611-a4a6-58dddcb808ad None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:57.929352 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8461f276-a695-4611-a4a6-58dddcb808ad None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:57.929617 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8461f276-a695-4611-a4a6-58dddcb808ad None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:57.930267 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8461f276-a695-4611-a4a6-58dddcb808ad None admin] RBAC: Adding query filter params (name=service) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:57.930564 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8461f276-a695-4611-a4a6-58dddcb808ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:57.931937 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8461f276-a695-4611-a4a6-58dddcb808ad None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:57.940288 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 299/584] 10.222.0.61 () {56 vars in 1103 bytes} [Mon Aug 22 19:05:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:57.940871 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8461f276-a695-4611-a4a6-58dddcb808ad None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:57.989900 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:57.990052 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133b788c-642b-4615-acc2-1dea65f9f59c None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:57.992173 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:58.023530 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5397a615-1592-4504-b78c-65fb113fa238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:58.024148 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5397a615-1592-4504-b78c-65fb113fa238 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:58.024423 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5397a615-1592-4504-b78c-65fb113fa238 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:58.024692 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5397a615-1592-4504-b78c-65fb113fa238 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:58.025409 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5397a615-1592-4504-b78c-65fb113fa238 None admin] RBAC: Adding query filter params (role.id=324bd04d55c34b61a5282c9f6ed73f46, scope.project.id=e07dc5ebdc504c6da706a2d21c0a3970, user.id=2824359491934d0dacaa21a718becee5) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:58.025712 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5397a615-1592-4504-b78c-65fb113fa238 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:58.027126 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5397a615-1592-4504-b78c-65fb113fa238 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:58.032827 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 286/585] 10.222.0.61 () {56 vars in 1374 bytes} [Mon Aug 22 19:05:57 2022] GET /identity/v3/role_assignments?user.id=2824359491934d0dacaa21a718becee5&scope.project.id=e07dc5ebdc504c6da706a2d21c0a3970&role.id=324bd04d55c34b61a5282c9f6ed73f46 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:58.033581 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5397a615-1592-4504-b78c-65fb113fa238 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:58.053463 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8461f276-a695-4611-a4a6-58dddcb808ad None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:58.093048 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a96d8d9-6e07-4019-b5d0-afdf6653c1ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:58.094249 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a96d8d9-6e07-4019-b5d0-afdf6653c1ff None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:58.094527 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a96d8d9-6e07-4019-b5d0-afdf6653c1ff None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:58.094799 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a96d8d9-6e07-4019-b5d0-afdf6653c1ff None admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:58.095510 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a96d8d9-6e07-4019-b5d0-afdf6653c1ff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:58.095843 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a96d8d9-6e07-4019-b5d0-afdf6653c1ff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:58.097321 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a96d8d9-6e07-4019-b5d0-afdf6653c1ff None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:58.116739 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 300/586] 10.222.0.61 () {56 vars in 1148 bytes} [Mon Aug 22 19:05:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:58.117719 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a96d8d9-6e07-4019-b5d0-afdf6653c1ff None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:58.154689 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5397a615-1592-4504-b78c-65fb113fa238 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:58.193408 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4deabb-3f35-43f1-998e-96bccf69d10f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:58.194051 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4deabb-3f35-43f1-998e-96bccf69d10f None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:58.194331 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4deabb-3f35-43f1-998e-96bccf69d10f None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:58.194604 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4deabb-3f35-43f1-998e-96bccf69d10f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:58.206036 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4deabb-3f35-43f1-998e-96bccf69d10f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:58.207472 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4deabb-3f35-43f1-998e-96bccf69d10f None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:58.222524 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-4b4deabb-3f35-43f1-998e-96bccf69d10f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:05:58.222524 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:05:58.222524 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:05:58.222524 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:05:58.222524 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:05:58.222524 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:05:58.222524 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:05:58.222524 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:05:58.222524 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:05:58.222524 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:05:58.222524 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:05:58.222524 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:05:58.222524 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 19:05:58.222524 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 19:05:58.222524 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 19:05:58.222524 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 19:05:58.223702 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:05:58.223702 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:05:58.223702 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:05:58.223702 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:05:58.223702 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:05:58.223702 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:05:58.223702 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:05:58.223702 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:05:58.223702 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:05:58.223702 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:05:58.223702 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:05:58.223702 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:05:58.223702 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:05:58.223702 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:05:58.223702 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:05:58.223702 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:05:58.223702 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 19:05:58.224605 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 19:05:58.224605 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:05:58.224605 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:05:58.224605 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 287/587] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:05:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:05:58.224835 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4deabb-3f35-43f1-998e-96bccf69d10f None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:58.234830 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a96d8d9-6e07-4019-b5d0-afdf6653c1ff None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:58.246296 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:58.254780 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:58.272681 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b40c432-30ce-4d8e-99a2-64575a2fca18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:58.273291 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b40c432-30ce-4d8e-99a2-64575a2fca18 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:58.273590 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b40c432-30ce-4d8e-99a2-64575a2fca18 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:58.273849 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b40c432-30ce-4d8e-99a2-64575a2fca18 None admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:58.274582 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b40c432-30ce-4d8e-99a2-64575a2fca18 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:58.274889 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b40c432-30ce-4d8e-99a2-64575a2fca18 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:58.276348 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b40c432-30ce-4d8e-99a2-64575a2fca18 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:58.294888 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 301/588] 10.222.0.61 () {56 vars in 1139 bytes} [Mon Aug 22 19:05:58 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:58.295700 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b40c432-30ce-4d8e-99a2-64575a2fca18 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:58.344889 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4deabb-3f35-43f1-998e-96bccf69d10f None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:58.380269 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f002741c-1848-467a-b44b-e7f81437de4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:58.380621 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f002741c-1848-467a-b44b-e7f81437de4d None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:58.380898 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f002741c-1848-467a-b44b-e7f81437de4d None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:58.381169 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f002741c-1848-467a-b44b-e7f81437de4d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:58.392346 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f002741c-1848-467a-b44b-e7f81437de4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:58.393789 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f002741c-1848-467a-b44b-e7f81437de4d None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:58.404063 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b40c432-30ce-4d8e-99a2-64575a2fca18 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:58.409168 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-f002741c-1848-467a-b44b-e7f81437de4d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:05:58.409168 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:05:58.409168 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:05:58.409168 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:05:58.409168 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:05:58.409168 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:05:58.409168 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:05:58.409168 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:05:58.409168 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:05:58.409168 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:05:58.409168 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:05:58.409168 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:05:58.409168 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 19:05:58.409168 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 19:05:58.409168 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 19:05:58.409168 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 19:05:58.410083 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:05:58.410083 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:05:58.410083 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:05:58.410083 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:05:58.410083 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:05:58.410083 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:05:58.410083 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:05:58.410083 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:05:58.410083 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:05:58.410083 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:05:58.410083 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:05:58.410083 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:05:58.410083 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:05:58.410083 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:05:58.410083 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:05:58.410083 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:05:58.410083 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 19:05:58.410974 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 19:05:58.410974 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:05:58.410974 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:05:58.410974 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 288/589] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:05:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:05:58.411201 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f002741c-1848-467a-b44b-e7f81437de4d None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:58.446243 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456377d8-6a8d-44ae-b8c6-541c70809b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:58.446902 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456377d8-6a8d-44ae-b8c6-541c70809b2a None admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:58.447175 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456377d8-6a8d-44ae-b8c6-541c70809b2a None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:58.447453 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456377d8-6a8d-44ae-b8c6-541c70809b2a None admin] PATH_INFO: `/v3/projects/e07dc5ebdc504c6da706a2d21c0a3970/users/e7f43e2756b746a6825ca1a06b7d7c91/roles/324bd04d55c34b61a5282c9f6ed73f46` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:58.464723 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456377d8-6a8d-44ae-b8c6-541c70809b2a None admin] RBAC: Authorizing `identity:create_grant(project_id=e07dc5ebdc504c6da706a2d21c0a3970, user_id=e7f43e2756b746a6825ca1a06b7d7c91, role_id=324bd04d55c34b61a5282c9f6ed73f46)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:58.466301 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456377d8-6a8d-44ae-b8c6-541c70809b2a None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:58.482293 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 302/590] 10.222.0.61 () {58 vars in 1439 bytes} [Mon Aug 22 19:05:58 2022] PUT /identity/v3/projects/e07dc5ebdc504c6da706a2d21c0a3970/users/e7f43e2756b746a6825ca1a06b7d7c91/roles/324bd04d55c34b61a5282c9f6ed73f46 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:05:58.483276 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456377d8-6a8d-44ae-b8c6-541c70809b2a None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:58.506491 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:58.515072 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:58.550316 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f002741c-1848-467a-b44b-e7f81437de4d None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:58.590800 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bcb711-811e-4580-bc0b-66e23154638a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:58.591820 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bcb711-811e-4580-bc0b-66e23154638a None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:58.592286 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bcb711-811e-4580-bc0b-66e23154638a None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:58.592829 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bcb711-811e-4580-bc0b-66e23154638a None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:58.594079 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bcb711-811e-4580-bc0b-66e23154638a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:58.594384 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bcb711-811e-4580-bc0b-66e23154638a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:58.595796 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bcb711-811e-4580-bc0b-66e23154638a None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:58.609742 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 289/591] 10.222.0.61 () {56 vars in 1109 bytes} [Mon Aug 22 19:05:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:58.610622 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bcb711-811e-4580-bc0b-66e23154638a None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:58.633822 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456377d8-6a8d-44ae-b8c6-541c70809b2a None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:58.681222 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf46f78-c560-451b-8f33-744d0bb36fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:58.682163 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf46f78-c560-451b-8f33-744d0bb36fde None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:58.682439 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf46f78-c560-451b-8f33-744d0bb36fde None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:58.682704 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf46f78-c560-451b-8f33-744d0bb36fde None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:58.697430 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf46f78-c560-451b-8f33-744d0bb36fde None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:58.699160 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf46f78-c560-451b-8f33-744d0bb36fde None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:58.714915 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-7cf46f78-c560-451b-8f33-744d0bb36fde None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:05:58.714915 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:05:58.714915 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:05:58.714915 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:05:58.714915 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:05:58.714915 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:05:58.714915 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:05:58.714915 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:05:58.714915 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:05:58.714915 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:05:58.714915 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:05:58.714915 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:05:58.714915 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 19:05:58.714915 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 19:05:58.714915 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 19:05:58.714915 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 19:05:58.715959 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:05:58.715959 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:05:58.715959 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:05:58.715959 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:05:58.715959 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:05:58.715959 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:05:58.715959 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:05:58.715959 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:05:58.715959 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:05:58.715959 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:05:58.715959 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:05:58.715959 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:05:58.715959 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:05:58.715959 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:05:58.715959 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:05:58.715959 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:05:58.715959 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 19:05:58.716865 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 19:05:58.716865 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:05:58.716865 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:05:58.716865 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 303/592] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:05:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:05:58.717103 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf46f78-c560-451b-8f33-744d0bb36fde None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:58.739847 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bcb711-811e-4580-bc0b-66e23154638a None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:58.780743 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:58.791070 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562c5f49-5716-42cc-90e3-979e6bd55373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:58.791705 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562c5f49-5716-42cc-90e3-979e6bd55373 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:58.791984 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562c5f49-5716-42cc-90e3-979e6bd55373 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:58.792314 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562c5f49-5716-42cc-90e3-979e6bd55373 None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:58.792885 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562c5f49-5716-42cc-90e3-979e6bd55373 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:58.793187 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562c5f49-5716-42cc-90e3-979e6bd55373 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:58.794567 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562c5f49-5716-42cc-90e3-979e6bd55373 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:58.800535 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:58.807407 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 290/593] 10.222.0.61 () {56 vars in 1109 bytes} [Mon Aug 22 19:05:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:58.808327 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562c5f49-5716-42cc-90e3-979e6bd55373 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:58.846420 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf46f78-c560-451b-8f33-744d0bb36fde None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:58.892531 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b04a0e-097e-4bc0-b333-3392e41baded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:58.893548 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b04a0e-097e-4bc0-b333-3392e41baded None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:58.894138 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b04a0e-097e-4bc0-b333-3392e41baded None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:58.894591 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b04a0e-097e-4bc0-b333-3392e41baded None admin] PATH_INFO: `/v3/projects/service` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:58.903485 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b04a0e-097e-4bc0-b333-3392e41baded None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:58.905838 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b04a0e-097e-4bc0-b333-3392e41baded None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:58.914031 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562c5f49-5716-42cc-90e3-979e6bd55373 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:58.920266 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-c3b04a0e-097e-4bc0-b333-3392e41baded None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 19:05:58.920266 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:05:58.920266 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:05:58.920266 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:05:58.920266 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:05:58.920266 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:05:58.920266 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:05:58.920266 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:05:58.920266 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:05:58.920266 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:05:58.920266 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:05:58.920266 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:05:58.920266 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 19:05:58.920266 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 19:05:58.920266 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 19:05:58.921125 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 19:05:58.921125 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:05:58.921125 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:05:58.921125 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:05:58.921125 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:05:58.921125 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:05:58.921125 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:05:58.921125 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:05:58.921125 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:05:58.921125 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:05:58.921125 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:05:58.921125 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:05:58.921125 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:05:58.921125 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:05:58.921125 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:05:58.921125 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:05:58.921125 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:05:58.922055 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 19:05:58.922055 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 19:05:58.922055 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 19:05:58.922055 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 19:05:58.922055 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 19:05:58.922055 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 19:05:58.922055 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 19:05:58.922055 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:05:58.924477 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 304/594] 10.222.0.61 () {56 vars in 1111 bytes} [Mon Aug 22 19:05:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:05:58.925906 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b04a0e-097e-4bc0-b333-3392e41baded None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:58.953987 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1ad1a5-bbdd-468c-a9f1-87d6fced208c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:58.954768 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1ad1a5-bbdd-468c-a9f1-87d6fced208c None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:58.955144 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1ad1a5-bbdd-468c-a9f1-87d6fced208c None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:58.955515 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1ad1a5-bbdd-468c-a9f1-87d6fced208c None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:58.956563 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1ad1a5-bbdd-468c-a9f1-87d6fced208c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:58.956563 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1ad1a5-bbdd-468c-a9f1-87d6fced208c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:58.958203 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1ad1a5-bbdd-468c-a9f1-87d6fced208c None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:58.976068 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 291/595] 10.222.0.61 () {56 vars in 1109 bytes} [Mon Aug 22 19:05:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:58.977168 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1ad1a5-bbdd-468c-a9f1-87d6fced208c None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:59.034446 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:59.063375 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b04a0e-097e-4bc0-b333-3392e41baded None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:59.086384 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1ad1a5-bbdd-468c-a9f1-87d6fced208c None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:59.104308 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:59.115553 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86f0067-a3f9-4e00-9058-54c5d12d321d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:59.116509 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86f0067-a3f9-4e00-9058-54c5d12d321d None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:59.116799 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86f0067-a3f9-4e00-9058-54c5d12d321d None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:59.117537 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86f0067-a3f9-4e00-9058-54c5d12d321d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:59.126234 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d7c955-a9ae-433a-b305-1b2b8f052e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:59.126234 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d7c955-a9ae-433a-b305-1b2b8f052e76 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:59.126634 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d7c955-a9ae-433a-b305-1b2b8f052e76 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:59.126806 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d7c955-a9ae-433a-b305-1b2b8f052e76 None admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:59.127516 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d7c955-a9ae-433a-b305-1b2b8f052e76 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:59.127854 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d7c955-a9ae-433a-b305-1b2b8f052e76 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:59.128752 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86f0067-a3f9-4e00-9058-54c5d12d321d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:59.129287 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d7c955-a9ae-433a-b305-1b2b8f052e76 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:59.130203 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86f0067-a3f9-4e00-9058-54c5d12d321d None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:59.146894 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-a86f0067-a3f9-4e00-9058-54c5d12d321d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 22 19:05:59.146894 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:05:59.146894 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:05:59.146894 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:05:59.146894 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:05:59.146894 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:05:59.146894 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:05:59.146894 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:05:59.146894 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:05:59.146894 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:05:59.146894 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:05:59.146894 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:05:59.146894 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 19:05:59.146894 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 19:05:59.146894 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 19:05:59.146894 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 19:05:59.148213 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:05:59.148213 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:05:59.148213 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 19:05:59.148213 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 19:05:59.148213 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 19:05:59.148213 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 19:05:59.148213 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:05:59.148213 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:05:59.148213 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:05:59.148213 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:05:59.148213 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:05:59.148213 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:05:59.148213 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:05:59.148213 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:05:59.148213 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:05:59.148213 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:05:59.148213 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:05:59.148213 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:05:59.149195 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:05:59.149195 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:05:59.149195 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 19:05:59.149195 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 19:05:59.149195 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 19:05:59.149195 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 19:05:59.149195 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 19:05:59.149195 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 19:05:59.149195 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 22 19:05:59.149195 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:05:59.154669 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 292/596] 10.222.0.61 () {56 vars in 1148 bytes} [Mon Aug 22 19:05:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:59.154669 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d7c955-a9ae-433a-b305-1b2b8f052e76 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:59.155181 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 305/597] 10.222.0.61 () {56 vars in 1099 bytes} [Mon Aug 22 19:05:58 2022] GET /identity/v3/users/cinder => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:05:59.155181 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86f0067-a3f9-4e00-9058-54c5d12d321d None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:59.262109 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d7c955-a9ae-433a-b305-1b2b8f052e76 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:59.267289 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86f0067-a3f9-4e00-9058-54c5d12d321d None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:59.276575 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:59.300167 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c801bd7-2622-4ed3-8c18-b16c5f2983f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:59.301133 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c801bd7-2622-4ed3-8c18-b16c5f2983f7 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:59.301635 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c801bd7-2622-4ed3-8c18-b16c5f2983f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:59.302218 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c801bd7-2622-4ed3-8c18-b16c5f2983f7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:59.305430 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab2aa7c-0064-4c56-a41f-38e2aae3d94c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:59.306191 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab2aa7c-0064-4c56-a41f-38e2aae3d94c None admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:59.306564 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab2aa7c-0064-4c56-a41f-38e2aae3d94c None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:59.306932 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab2aa7c-0064-4c56-a41f-38e2aae3d94c None admin] PATH_INFO: `/v3/projects/e07dc5ebdc504c6da706a2d21c0a3970/users/fe15d6a79ed44e5396d13cd29ba160ac/roles/324bd04d55c34b61a5282c9f6ed73f46` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:59.310576 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c801bd7-2622-4ed3-8c18-b16c5f2983f7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:59.312322 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c801bd7-2622-4ed3-8c18-b16c5f2983f7 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:59.321974 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab2aa7c-0064-4c56-a41f-38e2aae3d94c None admin] RBAC: Authorizing `identity:create_grant(project_id=e07dc5ebdc504c6da706a2d21c0a3970, user_id=fe15d6a79ed44e5396d13cd29ba160ac, role_id=324bd04d55c34b61a5282c9f6ed73f46)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:59.323387 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab2aa7c-0064-4c56-a41f-38e2aae3d94c None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:59.324578 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-9c801bd7-2622-4ed3-8c18-b16c5f2983f7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 19:05:59.324578 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:05:59.324578 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:05:59.324578 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:05:59.324578 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:05:59.324578 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:05:59.324578 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:05:59.324578 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:05:59.324578 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:05:59.324578 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:05:59.324578 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:05:59.324578 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:05:59.324578 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 19:05:59.324578 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 19:05:59.324578 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 19:05:59.325695 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 19:05:59.325695 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:05:59.325695 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:05:59.325695 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:05:59.325695 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:05:59.325695 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:05:59.325695 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:05:59.325695 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:05:59.325695 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:05:59.325695 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:05:59.325695 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:05:59.325695 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:05:59.325695 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:05:59.325695 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:05:59.325695 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:05:59.325695 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:05:59.325695 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:05:59.326625 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 19:05:59.326625 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 19:05:59.326625 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 19:05:59.326625 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 19:05:59.326625 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 19:05:59.326625 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 19:05:59.326625 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 19:05:59.326625 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:05:59.329123 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 293/598] 10.222.0.61 () {56 vars in 1111 bytes} [Mon Aug 22 19:05:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:05:59.330097 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c801bd7-2622-4ed3-8c18-b16c5f2983f7 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:59.351832 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 306/599] 10.222.0.61 () {58 vars in 1439 bytes} [Mon Aug 22 19:05:59 2022] PUT /identity/v3/projects/e07dc5ebdc504c6da706a2d21c0a3970/users/fe15d6a79ed44e5396d13cd29ba160ac/roles/324bd04d55c34b61a5282c9f6ed73f46 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 19:05:59.352937 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab2aa7c-0064-4c56-a41f-38e2aae3d94c None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:59.434805 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:59.447171 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c801bd7-2622-4ed3-8c18-b16c5f2983f7 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:59.489809 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71332755-e436-42f4-8101-86f1da04d6c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:59.490825 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71332755-e436-42f4-8101-86f1da04d6c7 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:59.491313 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71332755-e436-42f4-8101-86f1da04d6c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:59.491832 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71332755-e436-42f4-8101-86f1da04d6c7 None admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:59.492877 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71332755-e436-42f4-8101-86f1da04d6c7 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:59.493396 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71332755-e436-42f4-8101-86f1da04d6c7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:59.495260 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71332755-e436-42f4-8101-86f1da04d6c7 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:59.517355 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 294/600] 10.222.0.61 () {56 vars in 1137 bytes} [Mon Aug 22 19:05:59 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:59.523200 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71332755-e436-42f4-8101-86f1da04d6c7 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:59.549368 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab2aa7c-0064-4c56-a41f-38e2aae3d94c None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:59.557802 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:59.621100 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a1e4e5-a051-4b14-9f9a-912578a894ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:59.621764 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a1e4e5-a051-4b14-9f9a-912578a894ea None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:59.622042 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a1e4e5-a051-4b14-9f9a-912578a894ea None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:59.622315 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a1e4e5-a051-4b14-9f9a-912578a894ea None admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:59.623018 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a1e4e5-a051-4b14-9f9a-912578a894ea None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:59.623320 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a1e4e5-a051-4b14-9f9a-912578a894ea None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:59.625588 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a1e4e5-a051-4b14-9f9a-912578a894ea None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:59.638716 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71332755-e436-42f4-8101-86f1da04d6c7 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:59.652180 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 307/601] 10.222.0.61 () {56 vars in 1148 bytes} [Mon Aug 22 19:05:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:05:59.653632 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a1e4e5-a051-4b14-9f9a-912578a894ea None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:59.684088 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a83143a-cddd-48b6-b7a4-ee57a925af9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:59.685086 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a83143a-cddd-48b6-b7a4-ee57a925af9f None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:59.685569 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a83143a-cddd-48b6-b7a4-ee57a925af9f None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:59.686115 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a83143a-cddd-48b6-b7a4-ee57a925af9f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:59.698030 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a83143a-cddd-48b6-b7a4-ee57a925af9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:59.699829 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a83143a-cddd-48b6-b7a4-ee57a925af9f None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:59.716385 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-6a83143a-cddd-48b6-b7a4-ee57a925af9f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:05:59.716385 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:05:59.716385 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:05:59.716385 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:05:59.716385 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:05:59.716385 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:05:59.716385 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:05:59.716385 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:05:59.716385 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:05:59.716385 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:05:59.716385 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:05:59.716385 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:05:59.716385 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 19:05:59.716385 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 19:05:59.716385 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 19:05:59.716385 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 19:05:59.717369 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:05:59.717369 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:05:59.717369 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:05:59.717369 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:05:59.717369 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:05:59.717369 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:05:59.717369 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:05:59.717369 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:05:59.717369 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:05:59.717369 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:05:59.717369 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:05:59.717369 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:05:59.717369 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:05:59.717369 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:05:59.717369 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:05:59.717369 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:05:59.717369 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 19:05:59.718270 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 19:05:59.718270 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:05:59.718270 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:05:59.720454 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 295/602] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:05:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 19:05:59.726134 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a83143a-cddd-48b6-b7a4-ee57a925af9f None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:05:59.762965 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:59.767279 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a1e4e5-a051-4b14-9f9a-912578a894ea None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:59.816008 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4d838a-9a05-4331-8d4e-b8a2eb8c960b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:59.817235 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4d838a-9a05-4331-8d4e-b8a2eb8c960b None admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:59.817511 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4d838a-9a05-4331-8d4e-b8a2eb8c960b None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:59.817781 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4d838a-9a05-4331-8d4e-b8a2eb8c960b None admin] PATH_INFO: `/v3/projects/e07dc5ebdc504c6da706a2d21c0a3970/users/a4aca06c966a42f29f5c5bc108529d41/roles/324bd04d55c34b61a5282c9f6ed73f46` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:59.846887 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4d838a-9a05-4331-8d4e-b8a2eb8c960b None admin] RBAC: Authorizing `identity:create_grant(project_id=e07dc5ebdc504c6da706a2d21c0a3970, user_id=a4aca06c966a42f29f5c5bc108529d41, role_id=324bd04d55c34b61a5282c9f6ed73f46)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:59.851697 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4d838a-9a05-4331-8d4e-b8a2eb8c960b None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:05:59.861524 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:05:59.867691 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 308/603] 10.222.0.61 () {58 vars in 1439 bytes} [Mon Aug 22 19:05:59 2022] PUT /identity/v3/projects/e07dc5ebdc504c6da706a2d21c0a3970/users/a4aca06c966a42f29f5c5bc108529d41/roles/324bd04d55c34b61a5282c9f6ed73f46 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:05:59.916041 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a83143a-cddd-48b6-b7a4-ee57a925af9f None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:05:59.980190 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fb5e38-6d5a-4d12-8cfd-d0d110756e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:05:59.980815 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fb5e38-6d5a-4d12-8cfd-d0d110756e97 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:05:59.981089 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fb5e38-6d5a-4d12-8cfd-d0d110756e97 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:05:59.981360 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fb5e38-6d5a-4d12-8cfd-d0d110756e97 None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:05:59.982014 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fb5e38-6d5a-4d12-8cfd-d0d110756e97 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:05:59.982312 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fb5e38-6d5a-4d12-8cfd-d0d110756e97 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:05:59.983690 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fb5e38-6d5a-4d12-8cfd-d0d110756e97 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:00.001331 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 296/604] 10.222.0.61 () {56 vars in 1109 bytes} [Mon Aug 22 19:05:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:00.007191 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4d838a-9a05-4331-8d4e-b8a2eb8c960b None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:00.086524 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a4a074-35b4-4088-a2f6-3c0dee98672a None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:00.086702 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a4a074-35b4-4088-a2f6-3c0dee98672a None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:00.086972 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a4a074-35b4-4088-a2f6-3c0dee98672a None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:00.088956 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 297/605] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:06:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:06:00.094313 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4f5fe8-67c3-4b4c-8c8d-0b53ecea0087 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:00.094586 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4f5fe8-67c3-4b4c-8c8d-0b53ecea0087 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:00.094858 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4f5fe8-67c3-4b4c-8c8d-0b53ecea0087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:00.126351 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-da4f5fe8-67c3-4b4c-8c8d-0b53ecea0087 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:06:00.127841 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-da4f5fe8-67c3-4b4c-8c8d-0b53ecea0087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:06:00.172936 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4d838a-9a05-4331-8d4e-b8a2eb8c960b None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:00.218264 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07e4574-5b7e-441c-a1b6-7302a4da9b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:00.219431 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07e4574-5b7e-441c-a1b6-7302a4da9b74 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:00.219755 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07e4574-5b7e-441c-a1b6-7302a4da9b74 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:00.220455 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07e4574-5b7e-441c-a1b6-7302a4da9b74 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:00.228516 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07e4574-5b7e-441c-a1b6-7302a4da9b74 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:00.230727 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07e4574-5b7e-441c-a1b6-7302a4da9b74 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:00.241571 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 298/606] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:06:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 19:06:00.245466 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-c07e4574-5b7e-441c-a1b6-7302a4da9b74 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 19:06:00.245466 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:00.245466 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:00.245466 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:00.245466 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:00.245466 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:00.245466 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:00.245466 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:00.245466 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:00.245466 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:00.245466 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:00.245466 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:00.245466 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 19:06:00.245466 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 19:06:00.245466 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 19:06:00.246355 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 19:06:00.246355 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:00.246355 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:00.246355 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:00.246355 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:00.246355 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:00.246355 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:00.246355 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:00.246355 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:00.246355 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:00.246355 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:00.246355 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:00.246355 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:00.246355 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:00.246355 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:00.246355 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:00.246355 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:00.247288 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 19:06:00.247288 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 19:06:00.247288 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 19:06:00.247288 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 19:06:00.247288 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 19:06:00.247288 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 19:06:00.247288 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 19:06:00.247288 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:06:00.250755 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 309/607] 10.222.0.61 () {56 vars in 1111 bytes} [Mon Aug 22 19:06:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:00.257432 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4f5fe8-67c3-4b4c-8c8d-0b53ecea0087 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:00.342146 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:00.378655 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4f5fe8-67c3-4b4c-8c8d-0b53ecea0087 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:00.413846 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f666f1ea-22bc-4703-9188-05453bf25c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:00.414688 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f666f1ea-22bc-4703-9188-05453bf25c2f None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:00.415136 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f666f1ea-22bc-4703-9188-05453bf25c2f None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:00.415549 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f666f1ea-22bc-4703-9188-05453bf25c2f None admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:00.416412 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f666f1ea-22bc-4703-9188-05453bf25c2f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:00.416877 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f666f1ea-22bc-4703-9188-05453bf25c2f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:00.418441 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f666f1ea-22bc-4703-9188-05453bf25c2f None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:00.431251 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 299/608] 10.222.0.61 () {56 vars in 1148 bytes} [Mon Aug 22 19:06:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:00.437165 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07e4574-5b7e-441c-a1b6-7302a4da9b74 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:00.460003 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:00.544759 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07e4574-5b7e-441c-a1b6-7302a4da9b74 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:00.580342 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd57056-0ce7-4015-9259-fa80308d7136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:00.580998 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd57056-0ce7-4015-9259-fa80308d7136 None admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:00.581273 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd57056-0ce7-4015-9259-fa80308d7136 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:00.581545 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd57056-0ce7-4015-9259-fa80308d7136 None admin] PATH_INFO: `/v3/projects/e07dc5ebdc504c6da706a2d21c0a3970/users/56e79498873d4ed29a0e19bbea9b9214/roles/324bd04d55c34b61a5282c9f6ed73f46` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:00.596933 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd57056-0ce7-4015-9259-fa80308d7136 None admin] RBAC: Authorizing `identity:create_grant(project_id=e07dc5ebdc504c6da706a2d21c0a3970, user_id=56e79498873d4ed29a0e19bbea9b9214, role_id=324bd04d55c34b61a5282c9f6ed73f46)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:00.598287 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd57056-0ce7-4015-9259-fa80308d7136 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:00.616623 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3305c0-d63c-4fdb-bc5e-f0ef475c1d75 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:00.616945 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3305c0-d63c-4fdb-bc5e-f0ef475c1d75 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:00.617228 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3305c0-d63c-4fdb-bc5e-f0ef475c1d75 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:00.618912 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 300/609] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:06:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:06:00.627229 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3305c0-d63c-4fdb-bc5e-f0ef475c1d75 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:00.632295 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 310/610] 10.222.0.61 () {58 vars in 1439 bytes} [Mon Aug 22 19:06:00 2022] PUT /identity/v3/projects/e07dc5ebdc504c6da706a2d21c0a3970/users/56e79498873d4ed29a0e19bbea9b9214/roles/324bd04d55c34b61a5282c9f6ed73f46 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:06:00.633810 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cb9d0d-1b4e-4e52-9424-a3c0ee12cc04 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:00.634304 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cb9d0d-1b4e-4e52-9424-a3c0ee12cc04 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:00.634828 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cb9d0d-1b4e-4e52-9424-a3c0ee12cc04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:00.678114 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-44cb9d0d-1b4e-4e52-9424-a3c0ee12cc04 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:06:00.680001 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-44cb9d0d-1b4e-4e52-9424-a3c0ee12cc04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:06:00.803597 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:00.821793 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 311/611] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:06:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 19:06:00.825939 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3305c0-d63c-4fdb-bc5e-f0ef475c1d75 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:00.860215 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:00.865675 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236df8b9-67e0-43fb-9d4b-e2ea88d0d412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:00.866545 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236df8b9-67e0-43fb-9d4b-e2ea88d0d412 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:00.866822 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236df8b9-67e0-43fb-9d4b-e2ea88d0d412 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:00.867094 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236df8b9-67e0-43fb-9d4b-e2ea88d0d412 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:00.878475 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236df8b9-67e0-43fb-9d4b-e2ea88d0d412 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 19:06:00.883428 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-236df8b9-67e0-43fb-9d4b-e2ea88d0d412 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 19:06:00.883428 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:00.883428 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:00.883428 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:00.883428 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:00.883428 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:00.883428 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:00.883428 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:00.883428 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:00.883428 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:00.883428 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:00.883428 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:00.883428 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 19:06:00.883428 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 19:06:00.883428 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 19:06:00.883428 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 19:06:00.884391 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 19:06:00.884391 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 19:06:00.884391 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 19:06:00.884391 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 19:06:00.884391 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:00.884391 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:00.884391 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:00.884391 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:00.884391 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:00.884391 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:00.884391 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:00.884391 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:00.884391 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:00.884391 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:00.884391 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:00.884391 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:00.885244 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:00.885244 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:00.885244 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:00.885244 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:00.885244 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 19:06:00.885244 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 19:06:00.885244 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 19:06:00.885244 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 19:06:00.885244 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 19:06:00.885244 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 19:06:00.885244 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 19:06:00.885244 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:06:00.885244 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 301/612] 10.222.0.61 () {56 vars in 1102 bytes} [Mon Aug 22 19:06:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:00.892828 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cb9d0d-1b4e-4e52-9424-a3c0ee12cc04 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:01.045404 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cb9d0d-1b4e-4e52-9424-a3c0ee12cc04 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:01.097315 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987e9056-074b-4ff6-b508-7e11bb188cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:01.098271 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987e9056-074b-4ff6-b508-7e11bb188cc8 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:01.098734 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987e9056-074b-4ff6-b508-7e11bb188cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:01.099187 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987e9056-074b-4ff6-b508-7e11bb188cc8 None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:01.100059 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987e9056-074b-4ff6-b508-7e11bb188cc8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:01.100540 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987e9056-074b-4ff6-b508-7e11bb188cc8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:01.102099 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987e9056-074b-4ff6-b508-7e11bb188cc8 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:01.115618 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 312/613] 10.222.0.61 () {56 vars in 1103 bytes} [Mon Aug 22 19:06:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:01.121219 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236df8b9-67e0-43fb-9d4b-e2ea88d0d412 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:01.153468 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987e9056-074b-4ff6-b508-7e11bb188cc8 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:01.218232 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:01.315511 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987e9056-074b-4ff6-b508-7e11bb188cc8 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:01.333760 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236df8b9-67e0-43fb-9d4b-e2ea88d0d412 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:01.376706 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0647e8ef-623b-46be-9d47-d91ad3d4b771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:01.377310 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0647e8ef-623b-46be-9d47-d91ad3d4b771 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:01.377589 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0647e8ef-623b-46be-9d47-d91ad3d4b771 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:01.377857 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0647e8ef-623b-46be-9d47-d91ad3d4b771 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:01.386595 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0791473-39f4-41ad-8a74-883bd153e463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:01.387474 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0791473-39f4-41ad-8a74-883bd153e463 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:01.387919 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0791473-39f4-41ad-8a74-883bd153e463 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:01.388407 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0791473-39f4-41ad-8a74-883bd153e463 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:01.391666 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0647e8ef-623b-46be-9d47-d91ad3d4b771 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:01.391666 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0647e8ef-623b-46be-9d47-d91ad3d4b771 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:01.406256 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-0647e8ef-623b-46be-9d47-d91ad3d4b771 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:06:01.406256 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:01.406256 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:01.406256 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:01.406256 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:01.406256 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:01.406256 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:01.406256 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:01.406256 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:01.406256 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:01.406256 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:01.406256 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:01.406256 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 19:06:01.406256 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 19:06:01.406256 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 19:06:01.406256 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 19:06:01.407447 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:01.407447 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:01.407447 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:01.407447 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:01.407447 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:01.407447 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:01.407447 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:01.407447 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:01.407447 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:01.407447 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:01.407447 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:01.407447 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:01.407447 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:01.407447 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:01.407447 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:01.407447 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:01.407447 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 19:06:01.408551 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 19:06:01.408551 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:06:01.408551 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:06:01.409237 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0791473-39f4-41ad-8a74-883bd153e463 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 19:06:01.412428 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 302/614] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:06:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:01.413254 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0647e8ef-623b-46be-9d47-d91ad3d4b771 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:01.415953 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-b0791473-39f4-41ad-8a74-883bd153e463 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 19:06:01.415953 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:01.415953 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:01.415953 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:01.415953 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:01.415953 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:01.415953 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:01.415953 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:01.415953 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:01.415953 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:01.415953 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:01.415953 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:01.415953 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 19:06:01.415953 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 19:06:01.415953 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 19:06:01.415953 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 19:06:01.416889 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 19:06:01.416889 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 19:06:01.416889 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 19:06:01.416889 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 19:06:01.416889 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:01.416889 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:01.416889 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:01.416889 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:01.416889 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:01.416889 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:01.416889 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:01.416889 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:01.416889 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:01.416889 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:01.416889 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:01.416889 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:01.417749 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:01.417749 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:01.417749 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:01.417749 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:01.417749 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 19:06:01.417749 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 19:06:01.417749 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 19:06:01.417749 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 19:06:01.417749 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 19:06:01.417749 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 19:06:01.417749 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 19:06:01.417749 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:06:01.419963 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 313/615] 10.222.0.61 () {56 vars in 1102 bytes} [Mon Aug 22 19:06:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 19:06:01.429375 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0791473-39f4-41ad-8a74-883bd153e463 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:01.468125 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:01.555220 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0647e8ef-623b-46be-9d47-d91ad3d4b771 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:01.593685 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4f2127-8c9f-415d-b2f0-88d1aa2a7df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:01.594446 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4f2127-8c9f-415d-b2f0-88d1aa2a7df2 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:01.594828 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4f2127-8c9f-415d-b2f0-88d1aa2a7df2 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:01.595935 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4f2127-8c9f-415d-b2f0-88d1aa2a7df2 None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:01.597743 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4f2127-8c9f-415d-b2f0-88d1aa2a7df2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:01.598150 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4f2127-8c9f-415d-b2f0-88d1aa2a7df2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:01.601022 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4f2127-8c9f-415d-b2f0-88d1aa2a7df2 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:01.622801 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 303/616] 10.222.0.61 () {56 vars in 1109 bytes} [Mon Aug 22 19:06:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:01.628867 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4f2127-8c9f-415d-b2f0-88d1aa2a7df2 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:01.637612 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0791473-39f4-41ad-8a74-883bd153e463 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:01.663362 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:01.717238 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3c5df4-7af8-43cb-acbd-f5967cbd8893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:01.717850 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3c5df4-7af8-43cb-acbd-f5967cbd8893 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:01.718127 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3c5df4-7af8-43cb-acbd-f5967cbd8893 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:01.718946 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3c5df4-7af8-43cb-acbd-f5967cbd8893 None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:01.719614 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3c5df4-7af8-43cb-acbd-f5967cbd8893 None admin] RBAC: Adding query filter params (name=service) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:01.719967 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3c5df4-7af8-43cb-acbd-f5967cbd8893 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:01.721328 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3c5df4-7af8-43cb-acbd-f5967cbd8893 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:01.736374 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4f2127-8c9f-415d-b2f0-88d1aa2a7df2 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:01.738882 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:01.742080 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 314/617] 10.222.0.61 () {56 vars in 1103 bytes} [Mon Aug 22 19:06:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:01.747328 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3c5df4-7af8-43cb-acbd-f5967cbd8893 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:01.773705 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e94c751-7bf8-41e5-952d-073464828a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:01.774847 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e94c751-7bf8-41e5-952d-073464828a87 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:01.775233 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e94c751-7bf8-41e5-952d-073464828a87 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:01.775606 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e94c751-7bf8-41e5-952d-073464828a87 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:01.784312 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e94c751-7bf8-41e5-952d-073464828a87 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:01.785867 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e94c751-7bf8-41e5-952d-073464828a87 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:01.800007 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-3e94c751-7bf8-41e5-952d-073464828a87 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 22 19:06:01.800007 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:01.800007 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:01.800007 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:01.800007 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:01.800007 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:01.800007 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:01.800007 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:01.800007 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:01.800007 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:01.800007 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:01.800007 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:01.800007 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 19:06:01.800007 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 19:06:01.800007 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 19:06:01.800007 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 19:06:01.801159 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:01.801159 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:01.801159 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 19:06:01.801159 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 19:06:01.801159 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 19:06:01.801159 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 19:06:01.801159 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:01.801159 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:01.801159 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:01.801159 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:01.801159 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:01.801159 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:01.801159 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:01.801159 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:01.801159 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:01.801159 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:01.801159 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:01.801159 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:01.802414 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:01.802414 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:01.802414 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 19:06:01.802414 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 19:06:01.802414 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 19:06:01.802414 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 19:06:01.802414 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 19:06:01.802414 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 19:06:01.802414 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 22 19:06:01.802414 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:06:01.804545 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 304/618] 10.222.0.61 () {56 vars in 1096 bytes} [Mon Aug 22 19:06:01 2022] GET /identity/v3/users/swift => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:01.805752 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dd3da0-a50b-4617-9899-936a2562237b None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:01.806033 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dd3da0-a50b-4617-9899-936a2562237b None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:01.806310 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dd3da0-a50b-4617-9899-936a2562237b None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:01.807803 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 305/619] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:06:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:06:01.809664 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dd3da0-a50b-4617-9899-936a2562237b None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:01.908911 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3c5df4-7af8-43cb-acbd-f5967cbd8893 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:01.944284 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dd3da0-a50b-4617-9899-936a2562237b None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:01.950878 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6d1b3c-621a-480f-a41d-bfa0728e9f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:01.951540 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6d1b3c-621a-480f-a41d-bfa0728e9f48 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:01.951843 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6d1b3c-621a-480f-a41d-bfa0728e9f48 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:01.952113 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6d1b3c-621a-480f-a41d-bfa0728e9f48 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:01.967216 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6d1b3c-621a-480f-a41d-bfa0728e9f48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:01.969267 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6d1b3c-621a-480f-a41d-bfa0728e9f48 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:01.986281 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df8bb60-33d6-413f-8d97-56f8c2f24aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:01.986969 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df8bb60-33d6-413f-8d97-56f8c2f24aae None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:01.987313 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df8bb60-33d6-413f-8d97-56f8c2f24aae None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:01.987671 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df8bb60-33d6-413f-8d97-56f8c2f24aae None admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:01.987993 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-3d6d1b3c-621a-480f-a41d-bfa0728e9f48 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:06:01.987993 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:01.987993 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:01.987993 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:01.987993 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:01.987993 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:01.987993 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:01.987993 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:01.987993 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:01.987993 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:01.987993 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:01.987993 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:01.987993 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 19:06:01.987993 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 19:06:01.987993 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 19:06:01.987993 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 19:06:01.989357 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:01.989357 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:01.989357 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:01.989357 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:01.989357 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:01.989357 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:01.989357 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:01.989357 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:01.989357 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:01.989357 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:01.989357 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:01.989357 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:01.989357 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:01.989357 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:01.989357 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:01.989357 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:01.989357 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 19:06:01.990453 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 19:06:01.990453 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:06:01.990453 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:06:01.990453 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 315/620] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:06:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:01.995667 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e31dbd-7b31-437b-95de-00982d29171a None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:01.995667 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e31dbd-7b31-437b-95de-00982d29171a None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:01.995667 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e31dbd-7b31-437b-95de-00982d29171a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:01.995972 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df8bb60-33d6-413f-8d97-56f8c2f24aae None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:01.995972 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df8bb60-33d6-413f-8d97-56f8c2f24aae None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:01.996597 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df8bb60-33d6-413f-8d97-56f8c2f24aae None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:02.012928 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-c8e31dbd-7b31-437b-95de-00982d29171a None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:06:02.014819 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-c8e31dbd-7b31-437b-95de-00982d29171a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:06:02.019812 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 306/621] 10.222.0.61 () {56 vars in 1135 bytes} [Mon Aug 22 19:06:01 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:02.020669 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df8bb60-33d6-413f-8d97-56f8c2f24aae None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:02.043025 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:02.047201 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:02.097655 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 316/622] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:06:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 19:06:02.104846 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e31dbd-7b31-437b-95de-00982d29171a None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:02.136575 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df8bb60-33d6-413f-8d97-56f8c2f24aae None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:02.179481 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bf7fe1-31d7-42a5-991a-c2a4019d3a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:02.180119 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bf7fe1-31d7-42a5-991a-c2a4019d3a4b None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:02.180396 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bf7fe1-31d7-42a5-991a-c2a4019d3a4b None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:02.180664 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bf7fe1-31d7-42a5-991a-c2a4019d3a4b None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:02.181312 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bf7fe1-31d7-42a5-991a-c2a4019d3a4b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:02.181609 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bf7fe1-31d7-42a5-991a-c2a4019d3a4b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:02.183023 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bf7fe1-31d7-42a5-991a-c2a4019d3a4b None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:02.195476 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 307/623] 10.222.0.61 () {56 vars in 1109 bytes} [Mon Aug 22 19:06:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:02.208824 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bf7fe1-31d7-42a5-991a-c2a4019d3a4b None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:02.287942 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e31dbd-7b31-437b-95de-00982d29171a None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:02.356973 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e3beea-e1bf-42c4-a5fa-7ef07c840df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:02.357616 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e3beea-e1bf-42c4-a5fa-7ef07c840df3 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:02.357890 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e3beea-e1bf-42c4-a5fa-7ef07c840df3 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:02.358161 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e3beea-e1bf-42c4-a5fa-7ef07c840df3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:02.372225 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:02.375329 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e3beea-e1bf-42c4-a5fa-7ef07c840df3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:02.376832 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e3beea-e1bf-42c4-a5fa-7ef07c840df3 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:02.402517 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-84e3beea-e1bf-42c4-a5fa-7ef07c840df3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:06:02.402517 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:02.402517 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:02.402517 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:02.402517 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:02.402517 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:02.402517 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:02.402517 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:02.402517 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:02.402517 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:02.402517 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:02.402517 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:02.402517 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 19:06:02.402517 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 19:06:02.402517 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 19:06:02.402517 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 19:06:02.403509 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:02.403509 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:02.403509 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:02.403509 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:02.403509 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:02.403509 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:02.403509 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:02.403509 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:02.403509 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:02.403509 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:02.403509 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:02.403509 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:02.403509 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:02.403509 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:02.403509 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:02.403509 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:02.403509 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 19:06:02.404416 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 19:06:02.404416 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:06:02.404416 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:06:02.404416 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 317/624] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:06:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:02.411079 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bf7fe1-31d7-42a5-991a-c2a4019d3a4b None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:02.413354 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e3beea-e1bf-42c4-a5fa-7ef07c840df3 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:02.425434 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:02.454475 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dc7160-8f9e-4828-9f55-675a4e7df967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:02.455927 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dc7160-8f9e-4828-9f55-675a4e7df967 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:02.456205 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dc7160-8f9e-4828-9f55-675a4e7df967 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:02.456532 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dc7160-8f9e-4828-9f55-675a4e7df967 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:02.466332 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dc7160-8f9e-4828-9f55-675a4e7df967 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:02.469170 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dc7160-8f9e-4828-9f55-675a4e7df967 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:02.485113 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-74dc7160-8f9e-4828-9f55-675a4e7df967 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 22 19:06:02.485113 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:02.485113 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:02.485113 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:02.485113 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:02.485113 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:02.485113 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:02.485113 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:02.485113 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:02.485113 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:02.485113 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:02.485113 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:02.485113 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 19:06:02.485113 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 19:06:02.485113 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 19:06:02.485113 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 19:06:02.486135 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:02.486135 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:02.486135 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 19:06:02.486135 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 19:06:02.486135 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 19:06:02.486135 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 19:06:02.486135 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:02.486135 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:02.486135 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:02.486135 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:02.486135 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:02.486135 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:02.486135 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:02.486135 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:02.486135 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:02.486135 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:02.486135 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:02.486135 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:02.487086 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:02.487086 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:02.487086 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 19:06:02.487086 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 19:06:02.487086 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 19:06:02.487086 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 19:06:02.487086 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 19:06:02.487086 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 19:06:02.487086 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 22 19:06:02.487086 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:06:02.487086 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 308/625] 10.222.0.61 () {56 vars in 1093 bytes} [Mon Aug 22 19:06:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:02.488240 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba708ed-5cd0-4417-9656-2d9cf4983818 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:02.493936 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba708ed-5cd0-4417-9656-2d9cf4983818 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:02.493936 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba708ed-5cd0-4417-9656-2d9cf4983818 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:02.493936 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 309/626] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:06:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:06:02.493936 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba708ed-5cd0-4417-9656-2d9cf4983818 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:02.526423 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e3beea-e1bf-42c4-a5fa-7ef07c840df3 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:02.566233 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb28a7c4-fae0-4516-912f-e90071fd9962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:02.566998 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb28a7c4-fae0-4516-912f-e90071fd9962 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:02.567370 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb28a7c4-fae0-4516-912f-e90071fd9962 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:02.567768 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb28a7c4-fae0-4516-912f-e90071fd9962 None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:02.568516 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb28a7c4-fae0-4516-912f-e90071fd9962 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:02.569572 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb28a7c4-fae0-4516-912f-e90071fd9962 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:02.570390 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb28a7c4-fae0-4516-912f-e90071fd9962 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:02.583919 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 318/627] 10.222.0.61 () {56 vars in 1109 bytes} [Mon Aug 22 19:06:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:02.588946 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb28a7c4-fae0-4516-912f-e90071fd9962 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:02.601472 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba708ed-5cd0-4417-9656-2d9cf4983818 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:02.636354 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f665b2-7cd9-473c-820d-c5da2a808522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:02.637058 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f665b2-7cd9-473c-820d-c5da2a808522 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:02.637400 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f665b2-7cd9-473c-820d-c5da2a808522 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:02.637750 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f665b2-7cd9-473c-820d-c5da2a808522 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:02.649120 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f665b2-7cd9-473c-820d-c5da2a808522 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 19:06:02.654569 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-43f665b2-7cd9-473c-820d-c5da2a808522 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 19:06:02.654569 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:02.654569 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:02.654569 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:02.654569 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:02.654569 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:02.654569 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:02.654569 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:02.654569 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:02.654569 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:02.654569 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:02.654569 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:02.654569 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 19:06:02.654569 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 19:06:02.654569 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 19:06:02.654569 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 19:06:02.655778 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 19:06:02.655778 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 19:06:02.655778 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 19:06:02.655778 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 19:06:02.655778 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:02.655778 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:02.655778 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:02.655778 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:02.655778 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:02.655778 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:02.655778 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:02.655778 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:02.655778 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:02.655778 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:02.655778 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:02.655778 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:02.656889 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:02.656889 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:02.656889 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:02.656889 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:02.656889 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 19:06:02.656889 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 19:06:02.656889 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 19:06:02.656889 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 19:06:02.656889 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 19:06:02.656889 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 19:06:02.656889 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 19:06:02.656889 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:06:02.656889 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 310/628] 10.222.0.61 () {56 vars in 1102 bytes} [Mon Aug 22 19:06:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:02.657619 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87212fe-8925-4937-9e47-b4713d336609 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:02.657718 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87212fe-8925-4937-9e47-b4713d336609 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:02.657996 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87212fe-8925-4937-9e47-b4713d336609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:02.664272 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:02.664741 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:02.677417 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-d87212fe-8925-4937-9e47-b4713d336609 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:06:02.678699 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-d87212fe-8925-4937-9e47-b4713d336609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:06:02.705706 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb28a7c4-fae0-4516-912f-e90071fd9962 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:02.745815 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531fc0ee-2d3b-4469-88f3-c86bb95f8c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:02.746379 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 311/629] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:06:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 19:06:02.747077 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87212fe-8925-4937-9e47-b4713d336609 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:02.757011 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531fc0ee-2d3b-4469-88f3-c86bb95f8c05 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:02.757486 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531fc0ee-2d3b-4469-88f3-c86bb95f8c05 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:02.758363 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531fc0ee-2d3b-4469-88f3-c86bb95f8c05 None admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:02.759280 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531fc0ee-2d3b-4469-88f3-c86bb95f8c05 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:02.760198 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531fc0ee-2d3b-4469-88f3-c86bb95f8c05 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:02.762077 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531fc0ee-2d3b-4469-88f3-c86bb95f8c05 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:02.787580 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 319/630] 10.222.0.61 () {56 vars in 1133 bytes} [Mon Aug 22 19:06:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:02.788779 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531fc0ee-2d3b-4469-88f3-c86bb95f8c05 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:02.854931 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87212fe-8925-4937-9e47-b4713d336609 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:02.887344 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:02.893258 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531fc0ee-2d3b-4469-88f3-c86bb95f8c05 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:02.893775 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079a7cbf-da62-43ee-9040-b0ffc90f7d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:02.894407 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079a7cbf-da62-43ee-9040-b0ffc90f7d61 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:02.894800 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079a7cbf-da62-43ee-9040-b0ffc90f7d61 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:02.895075 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079a7cbf-da62-43ee-9040-b0ffc90f7d61 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:02.902781 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079a7cbf-da62-43ee-9040-b0ffc90f7d61 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:02.904367 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079a7cbf-da62-43ee-9040-b0ffc90f7d61 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:02.918431 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-079a7cbf-da62-43ee-9040-b0ffc90f7d61 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 19:06:02.918431 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:02.918431 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:02.918431 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:02.918431 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:02.918431 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:02.918431 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:02.918431 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:02.918431 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:02.918431 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:02.918431 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:02.918431 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:02.918431 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 19:06:02.918431 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 19:06:02.918431 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 19:06:02.919569 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 19:06:02.919569 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:02.919569 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:02.919569 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:02.919569 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:02.919569 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:02.919569 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:02.919569 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:02.919569 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:02.919569 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:02.919569 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:02.919569 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:02.919569 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:02.919569 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:02.919569 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:02.919569 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:02.919569 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:02.920588 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 19:06:02.920588 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 19:06:02.920588 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 19:06:02.920588 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 19:06:02.920588 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 19:06:02.920588 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 19:06:02.920588 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 19:06:02.920588 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:06:02.921054 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:02.921129 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 312/631] 10.222.0.61 () {56 vars in 1111 bytes} [Mon Aug 22 19:06:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:02.921129 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079a7cbf-da62-43ee-9040-b0ffc90f7d61 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:02.932525 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62322c0c-dd89-400f-83c1-cc60d01552d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:02.933135 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62322c0c-dd89-400f-83c1-cc60d01552d0 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:02.933412 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62322c0c-dd89-400f-83c1-cc60d01552d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:02.933681 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62322c0c-dd89-400f-83c1-cc60d01552d0 None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:02.934346 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62322c0c-dd89-400f-83c1-cc60d01552d0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:02.934646 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62322c0c-dd89-400f-83c1-cc60d01552d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:02.936027 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62322c0c-dd89-400f-83c1-cc60d01552d0 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:02.944109 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 320/632] 10.222.0.61 () {56 vars in 1103 bytes} [Mon Aug 22 19:06:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:02.945234 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824244fc-8337-4e03-b499-a8d7c5419fb0 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:02.945516 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824244fc-8337-4e03-b499-a8d7c5419fb0 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:02.945789 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824244fc-8337-4e03-b499-a8d7c5419fb0 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:02.947120 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 321/633] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:06:02 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:06:02.948460 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824244fc-8337-4e03-b499-a8d7c5419fb0 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:03.037843 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079a7cbf-da62-43ee-9040-b0ffc90f7d61 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:03.060891 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824244fc-8337-4e03-b499-a8d7c5419fb0 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:03.071594 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383f1167-7525-4e4b-aaba-1cbe0804c3d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:03.072268 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383f1167-7525-4e4b-aaba-1cbe0804c3d8 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:03.072558 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383f1167-7525-4e4b-aaba-1cbe0804c3d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:03.072812 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383f1167-7525-4e4b-aaba-1cbe0804c3d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:03.095366 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd542c83-7e2f-4391-b951-9b6f98722158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:03.096014 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd542c83-7e2f-4391-b951-9b6f98722158 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:03.096288 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd542c83-7e2f-4391-b951-9b6f98722158 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:03.096553 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd542c83-7e2f-4391-b951-9b6f98722158 None admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:03.097235 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd542c83-7e2f-4391-b951-9b6f98722158 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:03.097533 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd542c83-7e2f-4391-b951-9b6f98722158 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:03.098977 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd542c83-7e2f-4391-b951-9b6f98722158 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:03.110692 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 322/634] 10.222.0.61 () {56 vars in 1148 bytes} [Mon Aug 22 19:06:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:03.111849 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd542c83-7e2f-4391-b951-9b6f98722158 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:03.131481 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:03.142064 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383f1167-7525-4e4b-aaba-1cbe0804c3d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:03.143517 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383f1167-7525-4e4b-aaba-1cbe0804c3d8 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:03.162377 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:03.211351 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd542c83-7e2f-4391-b951-9b6f98722158 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:03.242755 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 313/635] 10.222.0.61 () {58 vars in 1282 bytes} [Mon Aug 22 19:06:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 19:06:03.243994 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db665308-31a8-4940-b5ab-996534089ab9 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:03.244278 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db665308-31a8-4940-b5ab-996534089ab9 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:03.244545 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db665308-31a8-4940-b5ab-996534089ab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:03.249438 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2bca85-a733-46f7-8241-7f013962aac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:03.250592 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2bca85-a733-46f7-8241-7f013962aac7 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:03.250876 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2bca85-a733-46f7-8241-7f013962aac7 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:03.251150 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2bca85-a733-46f7-8241-7f013962aac7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:03.261012 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2bca85-a733-46f7-8241-7f013962aac7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:03.261861 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-db665308-31a8-4940-b5ab-996534089ab9 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:06:03.262432 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2bca85-a733-46f7-8241-7f013962aac7 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:03.263047 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-db665308-31a8-4940-b5ab-996534089ab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:06:03.277160 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-8e2bca85-a733-46f7-8241-7f013962aac7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:06:03.277160 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:03.277160 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:03.277160 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:03.277160 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:03.277160 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:03.277160 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:03.277160 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:03.277160 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:03.277160 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:03.277160 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:03.277160 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:03.277160 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 19:06:03.277160 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 19:06:03.277160 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 19:06:03.277160 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 19:06:03.278130 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:03.278130 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:03.278130 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:03.278130 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:03.278130 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:03.278130 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:03.278130 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:03.278130 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:03.278130 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:03.278130 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:03.278130 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:03.278130 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:03.278130 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:03.278130 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:03.278130 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:03.278130 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:03.278130 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 19:06:03.279585 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 19:06:03.279585 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:06:03.279585 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:06:03.279585 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 323/636] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:06:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:03.279585 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2bca85-a733-46f7-8241-7f013962aac7 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:03.333340 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 314/637] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:06:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 19:06:03.334068 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db665308-31a8-4940-b5ab-996534089ab9 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:03.373162 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:03.377743 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:03.381088 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2bca85-a733-46f7-8241-7f013962aac7 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:03.416049 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9519726b-248a-41be-8484-4845880bf042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:03.416665 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9519726b-248a-41be-8484-4845880bf042 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:03.416940 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9519726b-248a-41be-8484-4845880bf042 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:03.417207 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9519726b-248a-41be-8484-4845880bf042 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:03.427299 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9519726b-248a-41be-8484-4845880bf042 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 19:06:03.433280 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-9519726b-248a-41be-8484-4845880bf042 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 19:06:03.433280 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:03.433280 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:03.433280 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:03.433280 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:03.433280 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:03.433280 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:03.433280 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:03.433280 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:03.433280 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:03.433280 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:03.433280 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:03.433280 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 19:06:03.433280 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 19:06:03.433280 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 19:06:03.433280 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 19:06:03.434231 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 19:06:03.434231 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 19:06:03.434231 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 19:06:03.434231 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 19:06:03.434231 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:03.434231 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:03.434231 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:03.434231 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:03.434231 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:03.434231 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:03.434231 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:03.434231 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:03.434231 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:03.434231 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:03.434231 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:03.434231 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:03.435094 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:03.435094 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:03.435094 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:03.435094 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:03.435094 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 19:06:03.435094 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 19:06:03.435094 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 19:06:03.435094 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 19:06:03.435094 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 19:06:03.435094 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 19:06:03.435094 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 19:06:03.435094 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:06:03.435094 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 324/638] 10.222.0.61 () {56 vars in 1102 bytes} [Mon Aug 22 19:06:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:03.435094 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9519726b-248a-41be-8484-4845880bf042 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:03.460259 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db665308-31a8-4940-b5ab-996534089ab9 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:03.496219 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01170a1-540e-46be-9987-f9b34d443ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:03.496884 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01170a1-540e-46be-9987-f9b34d443ca0 None admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:03.497166 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01170a1-540e-46be-9987-f9b34d443ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:03.497451 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01170a1-540e-46be-9987-f9b34d443ca0 None admin] PATH_INFO: `/v3/projects/e07dc5ebdc504c6da706a2d21c0a3970/users/2824359491934d0dacaa21a718becee5/roles/324bd04d55c34b61a5282c9f6ed73f46` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:03.512419 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01170a1-540e-46be-9987-f9b34d443ca0 None admin] RBAC: Authorizing `identity:create_grant(project_id=e07dc5ebdc504c6da706a2d21c0a3970, user_id=2824359491934d0dacaa21a718becee5, role_id=324bd04d55c34b61a5282c9f6ed73f46)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:03.513858 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01170a1-540e-46be-9987-f9b34d443ca0 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:03.527674 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 315/639] 10.222.0.61 () {58 vars in 1439 bytes} [Mon Aug 22 19:06:03 2022] PUT /identity/v3/projects/e07dc5ebdc504c6da706a2d21c0a3970/users/2824359491934d0dacaa21a718becee5/roles/324bd04d55c34b61a5282c9f6ed73f46 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:06:03.528566 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01170a1-540e-46be-9987-f9b34d443ca0 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:03.557224 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9519726b-248a-41be-8484-4845880bf042 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:03.604797 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:03.605004 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392f68b0-693a-4f57-91e1-54c87d75fee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:03.605663 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392f68b0-693a-4f57-91e1-54c87d75fee7 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:03.605801 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392f68b0-693a-4f57-91e1-54c87d75fee7 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:03.606128 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392f68b0-693a-4f57-91e1-54c87d75fee7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:03.619875 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392f68b0-693a-4f57-91e1-54c87d75fee7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:03.621405 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392f68b0-693a-4f57-91e1-54c87d75fee7 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:03.636905 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-392f68b0-693a-4f57-91e1-54c87d75fee7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:06:03.636905 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:03.636905 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:03.636905 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:03.636905 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:03.636905 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:03.636905 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:03.636905 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:03.636905 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:03.636905 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:03.636905 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:03.636905 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:03.636905 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 19:06:03.636905 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 19:06:03.636905 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 19:06:03.636905 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 19:06:03.637955 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:03.637955 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:03.637955 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:03.637955 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:03.637955 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:03.637955 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:03.637955 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:03.637955 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:03.637955 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:03.637955 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:03.637955 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:03.637955 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:03.637955 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:03.637955 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:03.637955 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:03.637955 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:03.637955 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 19:06:03.638877 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 19:06:03.638877 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:06:03.638877 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:06:03.638877 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 325/640] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:06:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:03.638877 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392f68b0-693a-4f57-91e1-54c87d75fee7 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:03.646073 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01170a1-540e-46be-9987-f9b34d443ca0 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:03.658511 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:03.693325 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810b5a29-b24a-4452-9613-b77ba68c51cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:03.693935 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810b5a29-b24a-4452-9613-b77ba68c51cc None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:03.694212 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810b5a29-b24a-4452-9613-b77ba68c51cc None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:03.694480 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810b5a29-b24a-4452-9613-b77ba68c51cc None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:03.695132 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810b5a29-b24a-4452-9613-b77ba68c51cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:03.695429 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810b5a29-b24a-4452-9613-b77ba68c51cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:03.696895 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810b5a29-b24a-4452-9613-b77ba68c51cc None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:03.709895 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 316/641] 10.222.0.61 () {56 vars in 1109 bytes} [Mon Aug 22 19:06:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:03.711164 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810b5a29-b24a-4452-9613-b77ba68c51cc None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:03.753759 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392f68b0-693a-4f57-91e1-54c87d75fee7 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:03.788910 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13677191-eee3-4342-92a0-a0ee1c4c3e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:03.789520 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13677191-eee3-4342-92a0-a0ee1c4c3e18 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:03.789788 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13677191-eee3-4342-92a0-a0ee1c4c3e18 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:03.790058 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13677191-eee3-4342-92a0-a0ee1c4c3e18 None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:03.790711 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13677191-eee3-4342-92a0-a0ee1c4c3e18 None admin] RBAC: Adding query filter params (name=service) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:03.791010 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13677191-eee3-4342-92a0-a0ee1c4c3e18 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:03.792392 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13677191-eee3-4342-92a0-a0ee1c4c3e18 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:03.801239 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 326/642] 10.222.0.61 () {56 vars in 1103 bytes} [Mon Aug 22 19:06:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:03.801861 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13677191-eee3-4342-92a0-a0ee1c4c3e18 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:03.834467 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810b5a29-b24a-4452-9613-b77ba68c51cc None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:03.869891 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:03.877200 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3951f228-ab73-4578-862a-24531d57960a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:03.877837 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3951f228-ab73-4578-862a-24531d57960a None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:03.878113 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3951f228-ab73-4578-862a-24531d57960a None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:03.878382 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3951f228-ab73-4578-862a-24531d57960a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:03.881333 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:03.889811 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3951f228-ab73-4578-862a-24531d57960a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 19:06:03.895126 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-3951f228-ab73-4578-862a-24531d57960a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 19:06:03.895126 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:03.895126 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:03.895126 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:03.895126 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:03.895126 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:03.895126 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:03.895126 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:03.895126 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:03.895126 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:03.895126 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:03.895126 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:03.895126 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 19:06:03.895126 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 19:06:03.895126 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 19:06:03.895126 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 19:06:03.896213 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 19:06:03.896213 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 19:06:03.896213 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 19:06:03.896213 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 19:06:03.896213 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:03.896213 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:03.896213 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:03.896213 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:03.896213 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:03.896213 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:03.896213 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:03.896213 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:03.896213 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:03.896213 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:03.896213 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:03.896213 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:03.897089 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:03.897089 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:03.897089 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:03.897089 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:03.897089 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 19:06:03.897089 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 19:06:03.897089 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 19:06:03.897089 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 19:06:03.897089 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 19:06:03.897089 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 19:06:03.897089 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 19:06:03.897089 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:06:03.901419 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 317/643] 10.222.0.61 () {56 vars in 1102 bytes} [Mon Aug 22 19:06:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:03.901710 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3951f228-ab73-4578-862a-24531d57960a None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:03.909970 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13677191-eee3-4342-92a0-a0ee1c4c3e18 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:03.946275 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3cd9cc-e568-4a0d-b13f-8242d1f2f15a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:03.947041 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3cd9cc-e568-4a0d-b13f-8242d1f2f15a None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:03.947417 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3cd9cc-e568-4a0d-b13f-8242d1f2f15a None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:03.947830 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3cd9cc-e568-4a0d-b13f-8242d1f2f15a None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:03.948586 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3cd9cc-e568-4a0d-b13f-8242d1f2f15a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:03.948992 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3cd9cc-e568-4a0d-b13f-8242d1f2f15a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:03.950463 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3cd9cc-e568-4a0d-b13f-8242d1f2f15a None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:03.963482 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 327/644] 10.222.0.61 () {56 vars in 1109 bytes} [Mon Aug 22 19:06:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:03.969024 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3cd9cc-e568-4a0d-b13f-8242d1f2f15a None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:04.007375 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3951f228-ab73-4578-862a-24531d57960a None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:04.042522 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b05ceef-8b76-4f1e-a7b8-8e5921276090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:04.043294 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b05ceef-8b76-4f1e-a7b8-8e5921276090 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:04.043715 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b05ceef-8b76-4f1e-a7b8-8e5921276090 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:04.044091 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b05ceef-8b76-4f1e-a7b8-8e5921276090 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:04.052404 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b05ceef-8b76-4f1e-a7b8-8e5921276090 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:04.053958 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b05ceef-8b76-4f1e-a7b8-8e5921276090 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:04.065391 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-3b05ceef-8b76-4f1e-a7b8-8e5921276090 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 22 19:06:04.065391 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:04.065391 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:04.065391 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:04.065391 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:04.065391 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:04.065391 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:04.065391 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:04.065391 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:04.065391 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:04.065391 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:04.065391 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:04.065391 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 19:06:04.065391 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 19:06:04.065391 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 19:06:04.065391 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 19:06:04.066522 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:04.066522 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:04.066522 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 19:06:04.066522 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 19:06:04.066522 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 19:06:04.066522 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 19:06:04.066522 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:04.066522 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:04.066522 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:04.066522 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:04.066522 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:04.066522 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:04.066522 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:04.066522 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:04.066522 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:04.066522 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:04.066522 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:04.066522 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:04.067899 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:04.067899 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:04.067899 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 19:06:04.067899 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 19:06:04.067899 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 19:06:04.067899 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 19:06:04.067899 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 19:06:04.067899 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 19:06:04.067899 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 22 19:06:04.067899 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:06:04.068576 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 318/645] 10.222.0.61 () {56 vars in 1099 bytes} [Mon Aug 22 19:06:03 2022] GET /identity/v3/users/glance => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:04.073150 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b05ceef-8b76-4f1e-a7b8-8e5921276090 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:04.078668 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3cd9cc-e568-4a0d-b13f-8242d1f2f15a None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:04.117615 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0062eab4-1124-4e80-8caa-f8e524fca25e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:04.118252 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0062eab4-1124-4e80-8caa-f8e524fca25e None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:04.118527 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0062eab4-1124-4e80-8caa-f8e524fca25e None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:04.118796 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0062eab4-1124-4e80-8caa-f8e524fca25e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:04.121738 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:04.131581 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0062eab4-1124-4e80-8caa-f8e524fca25e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:04.133587 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0062eab4-1124-4e80-8caa-f8e524fca25e None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:04.140016 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:04.152815 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-0062eab4-1124-4e80-8caa-f8e524fca25e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:06:04.152815 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:04.152815 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:04.152815 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:04.152815 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:04.152815 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:04.152815 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:04.152815 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:04.152815 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:04.152815 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:04.152815 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:04.152815 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:04.152815 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 19:06:04.152815 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 19:06:04.152815 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 19:06:04.152815 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 19:06:04.153966 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:04.153966 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:04.153966 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:04.153966 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:04.153966 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:04.153966 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:04.153966 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:04.153966 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:04.153966 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:04.153966 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:04.153966 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:04.153966 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:04.153966 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:04.153966 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:04.153966 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:04.153966 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:04.153966 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 19:06:04.154918 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 19:06:04.154918 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:06:04.154918 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:06:04.154918 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 328/646] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:06:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:04.155154 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0062eab4-1124-4e80-8caa-f8e524fca25e None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:04.199815 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b05ceef-8b76-4f1e-a7b8-8e5921276090 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:04.238171 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26117ccf-6e67-462e-955a-d6c8164ae7c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:04.238780 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26117ccf-6e67-462e-955a-d6c8164ae7c6 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:04.239308 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26117ccf-6e67-462e-955a-d6c8164ae7c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:04.239576 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26117ccf-6e67-462e-955a-d6c8164ae7c6 None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:04.240267 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26117ccf-6e67-462e-955a-d6c8164ae7c6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:04.240685 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26117ccf-6e67-462e-955a-d6c8164ae7c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:04.242152 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26117ccf-6e67-462e-955a-d6c8164ae7c6 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:04.254662 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 319/647] 10.222.0.61 () {56 vars in 1103 bytes} [Mon Aug 22 19:06:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:04.254662 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26117ccf-6e67-462e-955a-d6c8164ae7c6 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:04.264065 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0062eab4-1124-4e80-8caa-f8e524fca25e None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:04.298403 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7db8af-641f-4d16-b9ad-cff657825dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:04.299031 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7db8af-641f-4d16-b9ad-cff657825dab None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:04.299310 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7db8af-641f-4d16-b9ad-cff657825dab None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:04.299580 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7db8af-641f-4d16-b9ad-cff657825dab None admin] PATH_INFO: `/v3/projects/service` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:04.306908 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7db8af-641f-4d16-b9ad-cff657825dab None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:04.308375 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7db8af-641f-4d16-b9ad-cff657825dab None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:04.328522 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-5e7db8af-641f-4d16-b9ad-cff657825dab None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 19:06:04.328522 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:04.328522 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:04.328522 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:04.328522 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:04.328522 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:04.328522 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:04.328522 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:04.328522 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:04.328522 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:04.328522 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:04.328522 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:04.328522 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 19:06:04.328522 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 19:06:04.328522 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 19:06:04.329592 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 19:06:04.329592 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:04.329592 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:04.329592 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:04.329592 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:04.329592 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:04.329592 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:04.329592 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:04.329592 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:04.329592 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:04.329592 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:04.329592 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:04.329592 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:04.329592 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:04.329592 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:04.329592 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:04.329592 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:04.330527 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 19:06:04.330527 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 19:06:04.330527 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 19:06:04.330527 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 19:06:04.330527 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 19:06:04.330527 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 19:06:04.330527 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 19:06:04.330527 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:06:04.335761 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 329/648] 10.222.0.61 () {56 vars in 1111 bytes} [Mon Aug 22 19:06:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:04.336655 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7db8af-641f-4d16-b9ad-cff657825dab None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:04.360154 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26117ccf-6e67-462e-955a-d6c8164ae7c6 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:04.375449 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:04.389974 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:04.397371 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4c4275-97a8-4dc5-9645-428e0abcf3c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:04.397985 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4c4275-97a8-4dc5-9645-428e0abcf3c2 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:04.398255 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4c4275-97a8-4dc5-9645-428e0abcf3c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:04.398527 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4c4275-97a8-4dc5-9645-428e0abcf3c2 None admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:04.399251 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4c4275-97a8-4dc5-9645-428e0abcf3c2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:04.399552 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4c4275-97a8-4dc5-9645-428e0abcf3c2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:04.400989 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4c4275-97a8-4dc5-9645-428e0abcf3c2 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:04.418056 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 320/649] 10.222.0.61 () {56 vars in 1137 bytes} [Mon Aug 22 19:06:04 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:04.423219 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4c4275-97a8-4dc5-9645-428e0abcf3c2 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:04.455077 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7db8af-641f-4d16-b9ad-cff657825dab None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:04.494635 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff2e026-a41b-49dd-8f3b-4a49cf84b934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:04.495397 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff2e026-a41b-49dd-8f3b-4a49cf84b934 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:04.495817 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff2e026-a41b-49dd-8f3b-4a49cf84b934 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:04.496195 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff2e026-a41b-49dd-8f3b-4a49cf84b934 None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:04.496954 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff2e026-a41b-49dd-8f3b-4a49cf84b934 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:04.497372 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff2e026-a41b-49dd-8f3b-4a49cf84b934 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:04.498838 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff2e026-a41b-49dd-8f3b-4a49cf84b934 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:04.512151 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 330/650] 10.222.0.61 () {56 vars in 1109 bytes} [Mon Aug 22 19:06:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:04.516750 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff2e026-a41b-49dd-8f3b-4a49cf84b934 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:04.529310 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4c4275-97a8-4dc5-9645-428e0abcf3c2 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:04.564592 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db883fb-9abd-422b-ad85-5df8ce7391d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:04.565373 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db883fb-9abd-422b-ad85-5df8ce7391d5 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:04.565754 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db883fb-9abd-422b-ad85-5df8ce7391d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:04.566128 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db883fb-9abd-422b-ad85-5df8ce7391d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:04.576444 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db883fb-9abd-422b-ad85-5df8ce7391d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:04.578045 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db883fb-9abd-422b-ad85-5df8ce7391d5 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:04.593841 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-9db883fb-9abd-422b-ad85-5df8ce7391d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:06:04.593841 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:04.593841 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:04.593841 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:04.593841 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:04.593841 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:04.593841 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:04.593841 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:04.593841 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:04.593841 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:04.593841 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:04.593841 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:04.593841 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 19:06:04.593841 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 19:06:04.593841 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 19:06:04.593841 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 19:06:04.595033 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:04.595033 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:04.595033 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:04.595033 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:04.595033 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:04.595033 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:04.595033 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:04.595033 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:04.595033 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:04.595033 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:04.595033 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:04.595033 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:04.595033 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:04.595033 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:04.595033 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:04.595033 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:04.595033 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 19:06:04.596192 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 19:06:04.596192 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:06:04.596192 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:06:04.597098 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 321/651] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:06:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:04.597907 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db883fb-9abd-422b-ad85-5df8ce7391d5 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:04.626763 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff2e026-a41b-49dd-8f3b-4a49cf84b934 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:04.631347 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:04.635577 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:04.666985 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cfde60-849f-45b8-899c-54d1f1f4cb3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:04.667594 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cfde60-849f-45b8-899c-54d1f1f4cb3f None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:04.667913 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cfde60-849f-45b8-899c-54d1f1f4cb3f None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:04.668173 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cfde60-849f-45b8-899c-54d1f1f4cb3f None admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:04.668868 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cfde60-849f-45b8-899c-54d1f1f4cb3f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:04.669222 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cfde60-849f-45b8-899c-54d1f1f4cb3f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:04.670595 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cfde60-849f-45b8-899c-54d1f1f4cb3f None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:04.685022 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 331/652] 10.222.0.61 () {56 vars in 1148 bytes} [Mon Aug 22 19:06:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:04.685912 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cfde60-849f-45b8-899c-54d1f1f4cb3f None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:04.714613 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db883fb-9abd-422b-ad85-5df8ce7391d5 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:04.752565 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7c2301-08ca-4d91-9a18-8413628c358c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:04.753292 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7c2301-08ca-4d91-9a18-8413628c358c None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:04.753831 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7c2301-08ca-4d91-9a18-8413628c358c None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:04.754203 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7c2301-08ca-4d91-9a18-8413628c358c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:04.789675 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cfde60-849f-45b8-899c-54d1f1f4cb3f None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:04.825988 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a0bbe5-95c4-4ef6-9642-666f656e1f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:04.826674 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a0bbe5-95c4-4ef6-9642-666f656e1f46 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:04.826957 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a0bbe5-95c4-4ef6-9642-666f656e1f46 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:04.827225 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a0bbe5-95c4-4ef6-9642-666f656e1f46 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:04.829946 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7c2301-08ca-4d91-9a18-8413628c358c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:04.831396 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7c2301-08ca-4d91-9a18-8413628c358c None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:04.836155 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a0bbe5-95c4-4ef6-9642-666f656e1f46 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:04.837594 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a0bbe5-95c4-4ef6-9642-666f656e1f46 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:04.849794 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-12a0bbe5-95c4-4ef6-9642-666f656e1f46 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 22 19:06:04.849794 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:04.849794 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:04.849794 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:04.849794 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:04.849794 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:04.849794 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:04.849794 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:04.849794 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:04.849794 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:04.849794 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:04.849794 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:04.849794 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 19:06:04.849794 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 19:06:04.849794 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 19:06:04.849794 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 19:06:04.850759 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:04.850759 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:04.850759 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 19:06:04.850759 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 19:06:04.850759 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 19:06:04.850759 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 19:06:04.850759 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:04.850759 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:04.850759 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:04.850759 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:04.850759 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:04.850759 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:04.850759 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:04.850759 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:04.850759 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:04.850759 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:04.850759 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:04.850759 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:04.852315 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:04.852315 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:04.852315 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 19:06:04.852315 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 19:06:04.852315 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 19:06:04.852315 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 19:06:04.852315 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 19:06:04.852315 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 19:06:04.852315 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 22 19:06:04.852315 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:06:04.852315 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 332/653] 10.222.0.61 () {56 vars in 1102 bytes} [Mon Aug 22 19:06:04 2022] GET /identity/v3/users/neutron => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:04.852315 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a0bbe5-95c4-4ef6-9642-666f656e1f46 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:04.861740 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:04.884410 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:04.940138 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 322/654] 10.222.0.61 () {58 vars in 1282 bytes} [Mon Aug 22 19:06:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 19:06:04.941242 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7c2301-08ca-4d91-9a18-8413628c358c None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:04.971165 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a0bbe5-95c4-4ef6-9642-666f656e1f46 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:05.011748 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232209f6-33cb-479a-bb68-ad95b6f83625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:05.012508 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232209f6-33cb-479a-bb68-ad95b6f83625 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:05.012756 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232209f6-33cb-479a-bb68-ad95b6f83625 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:05.013029 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232209f6-33cb-479a-bb68-ad95b6f83625 None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:05.014084 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232209f6-33cb-479a-bb68-ad95b6f83625 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:05.014197 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232209f6-33cb-479a-bb68-ad95b6f83625 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:05.015572 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232209f6-33cb-479a-bb68-ad95b6f83625 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:05.033203 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 333/655] 10.222.0.61 () {56 vars in 1109 bytes} [Mon Aug 22 19:06:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:05.033203 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232209f6-33cb-479a-bb68-ad95b6f83625 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:05.053450 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7c2301-08ca-4d91-9a18-8413628c358c None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:05.078325 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:05.090263 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ef7e1f-d0f2-4480-892f-f22a30eb8221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:05.090955 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ef7e1f-d0f2-4480-892f-f22a30eb8221 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:05.091307 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ef7e1f-d0f2-4480-892f-f22a30eb8221 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:05.091669 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ef7e1f-d0f2-4480-892f-f22a30eb8221 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:05.092505 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ef7e1f-d0f2-4480-892f-f22a30eb8221 None admin] RBAC: Adding query filter params (role.id=324bd04d55c34b61a5282c9f6ed73f46, scope.project.id=e07dc5ebdc504c6da706a2d21c0a3970, user.id=e7f43e2756b746a6825ca1a06b7d7c91) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:05.092811 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ef7e1f-d0f2-4480-892f-f22a30eb8221 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:05.094816 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ef7e1f-d0f2-4480-892f-f22a30eb8221 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:05.100807 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 323/656] 10.222.0.61 () {56 vars in 1374 bytes} [Mon Aug 22 19:06:04 2022] GET /identity/v3/role_assignments?user.id=e7f43e2756b746a6825ca1a06b7d7c91&scope.project.id=e07dc5ebdc504c6da706a2d21c0a3970&role.id=324bd04d55c34b61a5282c9f6ed73f46 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:05.101590 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ef7e1f-d0f2-4480-892f-f22a30eb8221 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:05.147997 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:05.166927 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232209f6-33cb-479a-bb68-ad95b6f83625 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:05.215954 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc849734-ca73-4ce4-a798-4ab5b11d215f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:05.216873 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc849734-ca73-4ce4-a798-4ab5b11d215f None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:05.217305 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc849734-ca73-4ce4-a798-4ab5b11d215f None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:05.218287 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc849734-ca73-4ce4-a798-4ab5b11d215f None admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:05.219817 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc849734-ca73-4ce4-a798-4ab5b11d215f None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:05.220585 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc849734-ca73-4ce4-a798-4ab5b11d215f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:05.222932 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc849734-ca73-4ce4-a798-4ab5b11d215f None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:05.231936 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ef7e1f-d0f2-4480-892f-f22a30eb8221 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:05.248778 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 334/657] 10.222.0.61 () {56 vars in 1139 bytes} [Mon Aug 22 19:06:05 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:05.250043 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc849734-ca73-4ce4-a798-4ab5b11d215f None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:05.277134 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0319592d-adcd-44ae-a0a4-0073dff363ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:05.277387 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0319592d-adcd-44ae-a0a4-0073dff363ea None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:05.277882 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0319592d-adcd-44ae-a0a4-0073dff363ea None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:05.278224 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0319592d-adcd-44ae-a0a4-0073dff363ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:05.293841 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0319592d-adcd-44ae-a0a4-0073dff363ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:05.296258 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0319592d-adcd-44ae-a0a4-0073dff363ea None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:05.313112 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-0319592d-adcd-44ae-a0a4-0073dff363ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:06:05.313112 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:05.313112 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:05.313112 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:05.313112 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:05.313112 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:05.313112 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:05.313112 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:05.313112 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:05.313112 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:05.313112 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:05.313112 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:05.313112 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 19:06:05.313112 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 19:06:05.313112 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 19:06:05.313112 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 19:06:05.314135 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:05.314135 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:05.314135 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:05.314135 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:05.314135 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:05.314135 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:05.314135 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:05.314135 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:05.314135 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:05.314135 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:05.314135 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:05.314135 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:05.314135 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:05.314135 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:05.314135 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:05.314135 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:05.314135 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 19:06:05.316036 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 19:06:05.316036 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:06:05.316036 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:06:05.316036 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 324/658] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:06:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:05.316036 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0319592d-adcd-44ae-a0a4-0073dff363ea None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:05.377887 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:05.392302 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc849734-ca73-4ce4-a798-4ab5b11d215f None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:05.429086 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f87561-b0ed-4d16-a475-f257b43d74d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:05.430120 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f87561-b0ed-4d16-a475-f257b43d74d1 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:05.430608 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f87561-b0ed-4d16-a475-f257b43d74d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:05.431105 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f87561-b0ed-4d16-a475-f257b43d74d1 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:05.439074 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f87561-b0ed-4d16-a475-f257b43d74d1 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:05.440813 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f87561-b0ed-4d16-a475-f257b43d74d1 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:05.453867 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-26f87561-b0ed-4d16-a475-f257b43d74d1 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 22 19:06:05.453867 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:05.453867 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:05.453867 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:05.453867 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:05.453867 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:05.453867 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:05.453867 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:05.453867 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:05.453867 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:05.453867 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:05.453867 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:05.453867 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 19:06:05.453867 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 19:06:05.453867 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 19:06:05.453867 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 19:06:05.454878 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:05.454878 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:05.454878 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 19:06:05.454878 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 19:06:05.454878 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 19:06:05.454878 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 19:06:05.454878 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:05.454878 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:05.454878 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:05.454878 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:05.454878 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:05.454878 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:05.454878 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:05.454878 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:05.454878 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:05.454878 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:05.454878 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:05.454878 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:05.456025 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:05.456025 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:05.456025 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 19:06:05.456025 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 19:06:05.456025 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 19:06:05.456025 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 19:06:05.456025 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 19:06:05.456025 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 19:06:05.456025 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 22 19:06:05.456025 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:06:05.456592 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0319592d-adcd-44ae-a0a4-0073dff363ea None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:05.456675 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 335/659] 10.222.0.61 () {56 vars in 1099 bytes} [Mon Aug 22 19:06:05 2022] GET /identity/v3/users/cinder => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:05.456675 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f87561-b0ed-4d16-a475-f257b43d74d1 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:05.460830 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:05.497913 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f738658-703e-4fe2-abb4-f5a2321ab60c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:05.498524 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f738658-703e-4fe2-abb4-f5a2321ab60c None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:05.498801 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f738658-703e-4fe2-abb4-f5a2321ab60c None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:05.499069 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f738658-703e-4fe2-abb4-f5a2321ab60c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:05.565160 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f87561-b0ed-4d16-a475-f257b43d74d1 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:05.570438 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f738658-703e-4fe2-abb4-f5a2321ab60c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:05.571959 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f738658-703e-4fe2-abb4-f5a2321ab60c None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:05.602261 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e97efa-6aa0-44bc-8aff-96976d177ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:05.602873 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e97efa-6aa0-44bc-8aff-96976d177ba9 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:05.603151 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e97efa-6aa0-44bc-8aff-96976d177ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:05.603424 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e97efa-6aa0-44bc-8aff-96976d177ba9 None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:05.604100 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e97efa-6aa0-44bc-8aff-96976d177ba9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:05.604401 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e97efa-6aa0-44bc-8aff-96976d177ba9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:05.606033 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e97efa-6aa0-44bc-8aff-96976d177ba9 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:05.612292 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:05.621242 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 336/660] 10.222.0.61 () {56 vars in 1109 bytes} [Mon Aug 22 19:06:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:05.622964 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af89792-d688-4980-852d-e4668b2152b6 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:05.623386 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af89792-d688-4980-852d-e4668b2152b6 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:05.623819 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af89792-d688-4980-852d-e4668b2152b6 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:05.626459 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 337/661] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:06:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:06:05.627182 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af89792-d688-4980-852d-e4668b2152b6 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:05.684602 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 325/662] 10.222.0.61 () {58 vars in 1282 bytes} [Mon Aug 22 19:06:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 19:06:05.685936 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f738658-703e-4fe2-abb4-f5a2321ab60c None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:05.697945 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:05.738547 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af89792-d688-4980-852d-e4668b2152b6 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:05.774628 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8407ddf9-bd1c-4653-bb30-9f45d6db4d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:05.775241 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8407ddf9-bd1c-4653-bb30-9f45d6db4d18 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:05.775518 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8407ddf9-bd1c-4653-bb30-9f45d6db4d18 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:05.775821 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8407ddf9-bd1c-4653-bb30-9f45d6db4d18 None admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:05.776550 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8407ddf9-bd1c-4653-bb30-9f45d6db4d18 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:05.776850 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8407ddf9-bd1c-4653-bb30-9f45d6db4d18 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:05.778276 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8407ddf9-bd1c-4653-bb30-9f45d6db4d18 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:05.796335 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f738658-703e-4fe2-abb4-f5a2321ab60c None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:05.797811 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 338/663] 10.222.0.61 () {56 vars in 1137 bytes} [Mon Aug 22 19:06:05 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:05.799703 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8040255-9da1-4a2a-894f-af72f408d621 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:05.800048 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8040255-9da1-4a2a-894f-af72f408d621 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:05.800329 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8040255-9da1-4a2a-894f-af72f408d621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:05.819018 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-d8040255-9da1-4a2a-894f-af72f408d621 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:06:05.820391 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-d8040255-9da1-4a2a-894f-af72f408d621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:06:05.839733 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c43a9d-e2de-4ae5-b4a6-2ea0c6a9684d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:05.840367 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c43a9d-e2de-4ae5-b4a6-2ea0c6a9684d None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:05.840638 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c43a9d-e2de-4ae5-b4a6-2ea0c6a9684d None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:05.840904 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c43a9d-e2de-4ae5-b4a6-2ea0c6a9684d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:05.848469 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c43a9d-e2de-4ae5-b4a6-2ea0c6a9684d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:05.849896 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c43a9d-e2de-4ae5-b4a6-2ea0c6a9684d None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:05.853345 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:05.863285 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-61c43a9d-e2de-4ae5-b4a6-2ea0c6a9684d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 19:06:05.863285 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:05.863285 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:05.863285 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:05.863285 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:05.863285 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:05.863285 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:05.863285 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:05.863285 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:05.863285 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:05.863285 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:05.863285 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:05.863285 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 19:06:05.863285 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 19:06:05.863285 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 19:06:05.864700 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 19:06:05.864700 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:05.864700 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:05.864700 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:05.864700 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:05.864700 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:05.864700 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:05.864700 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:05.864700 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:05.864700 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:05.864700 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:05.864700 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:05.864700 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:05.864700 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:05.864700 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:05.864700 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:05.864700 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:05.865861 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 19:06:05.865861 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 19:06:05.865861 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 19:06:05.865861 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 19:06:05.865861 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 19:06:05.865861 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 19:06:05.865861 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 19:06:05.865861 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:06:05.873482 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 326/664] 10.222.0.61 () {56 vars in 1111 bytes} [Mon Aug 22 19:06:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:05.873482 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c43a9d-e2de-4ae5-b4a6-2ea0c6a9684d None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:05.898227 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 339/665] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:06:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 19:06:05.898891 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8040255-9da1-4a2a-894f-af72f408d621 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:05.966817 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:06.008851 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c43a9d-e2de-4ae5-b4a6-2ea0c6a9684d None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:06.017817 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8040255-9da1-4a2a-894f-af72f408d621 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:06.053798 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c571c6-d5c0-48f7-ab38-b627acce8514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:06.054414 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c571c6-d5c0-48f7-ab38-b627acce8514 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:06.054834 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c571c6-d5c0-48f7-ab38-b627acce8514 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:06.055107 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c571c6-d5c0-48f7-ab38-b627acce8514 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:06.056741 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3f17e3-8f1c-4ba1-8de4-50ac50edf489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:06.057850 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3f17e3-8f1c-4ba1-8de4-50ac50edf489 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:06.058333 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3f17e3-8f1c-4ba1-8de4-50ac50edf489 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:06.059734 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3f17e3-8f1c-4ba1-8de4-50ac50edf489 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:06.074144 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3f17e3-8f1c-4ba1-8de4-50ac50edf489 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:06.076315 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3f17e3-8f1c-4ba1-8de4-50ac50edf489 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:06.098711 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-2b3f17e3-8f1c-4ba1-8de4-50ac50edf489 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:06:06.098711 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:06.098711 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:06.098711 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:06.098711 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:06.098711 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:06.098711 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:06.098711 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:06.098711 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:06.098711 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:06.098711 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:06.098711 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:06.098711 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 19:06:06.098711 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 19:06:06.098711 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 19:06:06.098711 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 19:06:06.099743 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:06.099743 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:06.099743 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:06.099743 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:06.099743 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:06.099743 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:06.099743 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:06.099743 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:06.099743 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:06.099743 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:06.099743 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:06.099743 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:06.099743 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:06.099743 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:06.099743 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:06.099743 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:06.099743 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 19:06:06.100671 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 19:06:06.100671 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:06:06.100671 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:06:06.103788 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 327/666] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:06:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:06.104788 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3f17e3-8f1c-4ba1-8de4-50ac50edf489 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:06.139290 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c571c6-d5c0-48f7-ab38-b627acce8514 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:06.146330 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c571c6-d5c0-48f7-ab38-b627acce8514 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:06.176500 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:06.204724 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:06.226888 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3f17e3-8f1c-4ba1-8de4-50ac50edf489 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:06.261417 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da49dc8e-980f-4418-93a9-44f23b7ca28f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:06.262024 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da49dc8e-980f-4418-93a9-44f23b7ca28f None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:06.262298 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da49dc8e-980f-4418-93a9-44f23b7ca28f None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:06.262565 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da49dc8e-980f-4418-93a9-44f23b7ca28f None admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:06.267893 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 340/667] 10.222.0.61 () {58 vars in 1282 bytes} [Mon Aug 22 19:06:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 19:06:06.268031 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da49dc8e-980f-4418-93a9-44f23b7ca28f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:06.268031 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da49dc8e-980f-4418-93a9-44f23b7ca28f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:06.268212 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c571c6-d5c0-48f7-ab38-b627acce8514 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:06.268301 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da49dc8e-980f-4418-93a9-44f23b7ca28f None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:06.280059 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 328/668] 10.222.0.61 () {56 vars in 1148 bytes} [Mon Aug 22 19:06:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:06.287023 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da49dc8e-980f-4418-93a9-44f23b7ca28f None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:06.370279 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c571c6-d5c0-48f7-ab38-b627acce8514 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:06.400588 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da49dc8e-980f-4418-93a9-44f23b7ca28f None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:06.404566 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f595717-c0bb-407e-8900-a9940be67fd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:06.405169 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f595717-c0bb-407e-8900-a9940be67fd2 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:06.405442 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f595717-c0bb-407e-8900-a9940be67fd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:06.405731 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f595717-c0bb-407e-8900-a9940be67fd2 None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:06.406374 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f595717-c0bb-407e-8900-a9940be67fd2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:06.406674 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f595717-c0bb-407e-8900-a9940be67fd2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:06.408052 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f595717-c0bb-407e-8900-a9940be67fd2 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:06.416121 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:06.420843 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 341/669] 10.222.0.61 () {56 vars in 1109 bytes} [Mon Aug 22 19:06:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:06.421966 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f595717-c0bb-407e-8900-a9940be67fd2 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:06.441800 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:06.442231 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dea9f01-1a36-4f69-9c5b-85bd6802ce86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:06.443187 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dea9f01-1a36-4f69-9c5b-85bd6802ce86 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:06.443541 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dea9f01-1a36-4f69-9c5b-85bd6802ce86 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:06.443853 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dea9f01-1a36-4f69-9c5b-85bd6802ce86 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:06.462127 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dea9f01-1a36-4f69-9c5b-85bd6802ce86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 19:06:06.466787 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-0dea9f01-1a36-4f69-9c5b-85bd6802ce86 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 19:06:06.466787 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:06.466787 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:06.466787 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:06.466787 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:06.466787 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:06.466787 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:06.466787 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:06.466787 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:06.466787 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:06.466787 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:06.466787 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:06.466787 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 19:06:06.466787 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 19:06:06.466787 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 19:06:06.466787 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 19:06:06.467753 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 19:06:06.467753 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 19:06:06.467753 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 19:06:06.467753 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 19:06:06.467753 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:06.467753 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:06.467753 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:06.467753 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:06.467753 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:06.467753 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:06.467753 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:06.467753 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:06.467753 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:06.467753 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:06.467753 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:06.467753 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:06.473419 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:06.473419 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:06.473419 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:06.473419 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:06.473419 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 19:06:06.473419 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 19:06:06.473419 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 19:06:06.473419 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 19:06:06.473419 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 19:06:06.473419 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 19:06:06.473419 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 19:06:06.473419 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:06:06.473419 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 329/670] 10.222.0.61 () {56 vars in 1102 bytes} [Mon Aug 22 19:06:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:06.473419 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dea9f01-1a36-4f69-9c5b-85bd6802ce86 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:06.543755 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f595717-c0bb-407e-8900-a9940be67fd2 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:06.571065 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dea9f01-1a36-4f69-9c5b-85bd6802ce86 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:06.587795 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b730ad-f843-40fc-aa71-dc70db1a932e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:06.588695 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b730ad-f843-40fc-aa71-dc70db1a932e None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:06.589002 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b730ad-f843-40fc-aa71-dc70db1a932e None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:06.589275 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b730ad-f843-40fc-aa71-dc70db1a932e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:06.599795 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b730ad-f843-40fc-aa71-dc70db1a932e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:06.601227 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b730ad-f843-40fc-aa71-dc70db1a932e None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:06.606260 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a600f4-7598-47cc-804a-c299c4908b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:06.606859 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a600f4-7598-47cc-804a-c299c4908b89 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:06.607132 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a600f4-7598-47cc-804a-c299c4908b89 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:06.607398 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a600f4-7598-47cc-804a-c299c4908b89 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:06.608144 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a600f4-7598-47cc-804a-c299c4908b89 None admin] RBAC: Adding query filter params (role.id=324bd04d55c34b61a5282c9f6ed73f46, scope.project.id=e07dc5ebdc504c6da706a2d21c0a3970, user.id=fe15d6a79ed44e5396d13cd29ba160ac) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:06.608447 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a600f4-7598-47cc-804a-c299c4908b89 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:06.610377 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a600f4-7598-47cc-804a-c299c4908b89 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:06.621662 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-d2b730ad-f843-40fc-aa71-dc70db1a932e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:06:06.621662 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:06.621662 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:06.621662 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:06.621662 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:06.621662 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:06.621662 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:06.621662 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:06.621662 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:06.621662 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:06.621662 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:06.621662 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:06.621662 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 19:06:06.621662 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 19:06:06.621662 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 19:06:06.621662 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 19:06:06.623076 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:06.623076 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:06.623076 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:06.623076 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:06.623076 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:06.623076 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:06.623076 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:06.623076 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:06.623076 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:06.623076 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:06.623076 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:06.623076 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:06.623076 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:06.623076 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:06.623076 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:06.623076 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:06.623076 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 19:06:06.624153 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 19:06:06.624153 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:06:06.624153 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:06:06.624336 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 330/671] 10.222.0.61 () {56 vars in 1374 bytes} [Mon Aug 22 19:06:06 2022] GET /identity/v3/role_assignments?user.id=fe15d6a79ed44e5396d13cd29ba160ac&scope.project.id=e07dc5ebdc504c6da706a2d21c0a3970&role.id=324bd04d55c34b61a5282c9f6ed73f46 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:06.624336 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a600f4-7598-47cc-804a-c299c4908b89 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:06.625038 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 342/672] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:06:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:06.625038 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b730ad-f843-40fc-aa71-dc70db1a932e None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:06.675266 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:06.702524 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:06.750405 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b730ad-f843-40fc-aa71-dc70db1a932e None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:06.763855 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a600f4-7598-47cc-804a-c299c4908b89 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:06.792285 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf47d62-eb92-4ccb-94d9-f3f62f871d04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:06.793055 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf47d62-eb92-4ccb-94d9-f3f62f871d04 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:06.793424 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf47d62-eb92-4ccb-94d9-f3f62f871d04 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:06.793852 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf47d62-eb92-4ccb-94d9-f3f62f871d04 None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:06.794468 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf47d62-eb92-4ccb-94d9-f3f62f871d04 None admin] RBAC: Adding query filter params (name=service) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:06.794765 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf47d62-eb92-4ccb-94d9-f3f62f871d04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:06.797451 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf47d62-eb92-4ccb-94d9-f3f62f871d04 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:06.806922 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 343/673] 10.222.0.61 () {56 vars in 1103 bytes} [Mon Aug 22 19:06:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:06.808248 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf47d62-eb92-4ccb-94d9-f3f62f871d04 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:06.814104 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05209747-d56f-47a9-8992-bc9a4b4d9c01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:06.814759 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05209747-d56f-47a9-8992-bc9a4b4d9c01 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:06.815043 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05209747-d56f-47a9-8992-bc9a4b4d9c01 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:06.815313 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05209747-d56f-47a9-8992-bc9a4b4d9c01 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:06.823256 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05209747-d56f-47a9-8992-bc9a4b4d9c01 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:06.824833 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05209747-d56f-47a9-8992-bc9a4b4d9c01 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:06.837718 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-05209747-d56f-47a9-8992-bc9a4b4d9c01 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 19:06:06.837718 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:06.837718 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:06.837718 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:06.837718 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:06.837718 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:06.837718 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:06.837718 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:06.837718 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:06.837718 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:06.837718 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:06.837718 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:06.837718 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 19:06:06.837718 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 19:06:06.837718 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 19:06:06.838636 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 19:06:06.838636 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:06.838636 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:06.838636 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:06.838636 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:06.838636 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:06.838636 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:06.838636 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:06.838636 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:06.838636 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:06.838636 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:06.838636 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:06.838636 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:06.838636 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:06.838636 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:06.838636 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:06.838636 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:06.840349 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 19:06:06.840349 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 19:06:06.840349 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 19:06:06.840349 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 19:06:06.840349 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 19:06:06.840349 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 19:06:06.840349 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 19:06:06.840349 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:06:06.840349 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 331/674] 10.222.0.61 () {56 vars in 1111 bytes} [Mon Aug 22 19:06:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:06.840349 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05209747-d56f-47a9-8992-bc9a4b4d9c01 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:06.946636 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf47d62-eb92-4ccb-94d9-f3f62f871d04 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:06.962903 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:06.966555 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05209747-d56f-47a9-8992-bc9a4b4d9c01 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:06.981174 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:06.992861 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bfb796-d879-448c-8ef3-77a2efb89e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:06.993966 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bfb796-d879-448c-8ef3-77a2efb89e11 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:06.994487 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bfb796-d879-448c-8ef3-77a2efb89e11 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:06.994943 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bfb796-d879-448c-8ef3-77a2efb89e11 None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:06.995701 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bfb796-d879-448c-8ef3-77a2efb89e11 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:06.996284 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bfb796-d879-448c-8ef3-77a2efb89e11 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:06.998583 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bfb796-d879-448c-8ef3-77a2efb89e11 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:07.005800 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ec3374-df25-45f2-95e6-49681487ef15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:07.006516 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ec3374-df25-45f2-95e6-49681487ef15 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:07.006864 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ec3374-df25-45f2-95e6-49681487ef15 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:07.007200 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ec3374-df25-45f2-95e6-49681487ef15 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:07.014218 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 344/675] 10.222.0.61 () {56 vars in 1109 bytes} [Mon Aug 22 19:06:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:07.015690 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bfb796-d879-448c-8ef3-77a2efb89e11 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:07.023619 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ec3374-df25-45f2-95e6-49681487ef15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:07.025287 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ec3374-df25-45f2-95e6-49681487ef15 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:07.041336 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-93ec3374-df25-45f2-95e6-49681487ef15 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:06:07.041336 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:07.041336 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:07.041336 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:07.041336 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:07.041336 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:07.041336 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:07.041336 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:07.041336 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:07.041336 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:07.041336 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:07.041336 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:07.041336 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 19:06:07.041336 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 19:06:07.041336 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 19:06:07.041336 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 19:06:07.042536 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:07.042536 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:07.042536 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:07.042536 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:07.042536 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:07.042536 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:07.042536 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:07.042536 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:07.042536 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:07.042536 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:07.042536 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:07.042536 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:07.042536 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:07.042536 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:07.042536 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:07.042536 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:07.042536 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 19:06:07.043629 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 19:06:07.043629 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:06:07.043629 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:06:07.044991 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 332/676] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:06:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:07.045090 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ec3374-df25-45f2-95e6-49681487ef15 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:07.149171 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bfb796-d879-448c-8ef3-77a2efb89e11 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:07.164368 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ec3374-df25-45f2-95e6-49681487ef15 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:07.185575 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72226d6a-4975-43ed-bf03-ef787366ef79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:07.186393 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72226d6a-4975-43ed-bf03-ef787366ef79 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:07.186669 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72226d6a-4975-43ed-bf03-ef787366ef79 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:07.187491 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72226d6a-4975-43ed-bf03-ef787366ef79 None admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:07.188213 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72226d6a-4975-43ed-bf03-ef787366ef79 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:07.188524 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72226d6a-4975-43ed-bf03-ef787366ef79 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:07.190146 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72226d6a-4975-43ed-bf03-ef787366ef79 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:07.199331 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cff53c-0a44-4089-bcbf-cd2f9216cee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:07.200215 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cff53c-0a44-4089-bcbf-cd2f9216cee5 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:07.200675 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cff53c-0a44-4089-bcbf-cd2f9216cee5 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:07.201127 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cff53c-0a44-4089-bcbf-cd2f9216cee5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:07.202244 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 345/677] 10.222.0.61 () {56 vars in 1148 bytes} [Mon Aug 22 19:06:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:07.203703 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72226d6a-4975-43ed-bf03-ef787366ef79 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:07.213854 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cff53c-0a44-4089-bcbf-cd2f9216cee5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:07.215339 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cff53c-0a44-4089-bcbf-cd2f9216cee5 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:07.226614 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-a6cff53c-0a44-4089-bcbf-cd2f9216cee5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 19:06:07.226614 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:07.226614 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:07.226614 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:07.226614 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:07.226614 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:07.226614 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:07.226614 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:07.226614 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:07.226614 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:07.226614 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:07.226614 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:07.226614 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 19:06:07.226614 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 19:06:07.226614 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 19:06:07.227735 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 19:06:07.227735 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:07.227735 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:07.227735 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:07.227735 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:07.227735 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:07.227735 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:07.227735 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:07.227735 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:07.227735 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:07.227735 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:07.227735 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:07.227735 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:07.227735 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:07.227735 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:07.227735 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:07.227735 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:07.228667 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 19:06:07.228667 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 19:06:07.228667 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 19:06:07.228667 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 19:06:07.228667 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 19:06:07.228667 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 19:06:07.228667 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 19:06:07.228667 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:06:07.228667 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 333/678] 10.222.0.61 () {56 vars in 1111 bytes} [Mon Aug 22 19:06:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:07.229161 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61005dad-dbc1-42fa-b016-c85826d6dbc9 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:07.229425 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61005dad-dbc1-42fa-b016-c85826d6dbc9 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:07.229695 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61005dad-dbc1-42fa-b016-c85826d6dbc9 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:07.234809 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 334/679] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:06:07 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:06:07.234809 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61005dad-dbc1-42fa-b016-c85826d6dbc9 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:07.244670 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:07.263809 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:07.326003 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72226d6a-4975-43ed-bf03-ef787366ef79 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:07.347744 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61005dad-dbc1-42fa-b016-c85826d6dbc9 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:07.361728 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de13671-fd7c-4f8e-a0f2-1262809638b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:07.362334 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de13671-fd7c-4f8e-a0f2-1262809638b6 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:07.362610 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de13671-fd7c-4f8e-a0f2-1262809638b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:07.362879 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de13671-fd7c-4f8e-a0f2-1262809638b6 None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:07.363519 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de13671-fd7c-4f8e-a0f2-1262809638b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:07.363845 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de13671-fd7c-4f8e-a0f2-1262809638b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:07.365458 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de13671-fd7c-4f8e-a0f2-1262809638b6 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:07.382521 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 346/680] 10.222.0.61 () {56 vars in 1109 bytes} [Mon Aug 22 19:06:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:07.383220 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de13671-fd7c-4f8e-a0f2-1262809638b6 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:07.384022 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37309a7e-f4ef-4e3d-b239-79f286203267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:07.384627 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37309a7e-f4ef-4e3d-b239-79f286203267 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:07.384900 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37309a7e-f4ef-4e3d-b239-79f286203267 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:07.385165 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37309a7e-f4ef-4e3d-b239-79f286203267 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:07.386053 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37309a7e-f4ef-4e3d-b239-79f286203267 None admin] RBAC: Adding query filter params (role.id=324bd04d55c34b61a5282c9f6ed73f46, scope.project.id=e07dc5ebdc504c6da706a2d21c0a3970, user.id=a4aca06c966a42f29f5c5bc108529d41) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:07.387184 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37309a7e-f4ef-4e3d-b239-79f286203267 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:07.388659 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37309a7e-f4ef-4e3d-b239-79f286203267 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:07.403691 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 335/681] 10.222.0.61 () {56 vars in 1374 bytes} [Mon Aug 22 19:06:07 2022] GET /identity/v3/role_assignments?user.id=a4aca06c966a42f29f5c5bc108529d41&scope.project.id=e07dc5ebdc504c6da706a2d21c0a3970&role.id=324bd04d55c34b61a5282c9f6ed73f46 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:07.403691 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b72f68d-584a-4213-b865-fc759eadd98e None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:07.403691 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b72f68d-584a-4213-b865-fc759eadd98e None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:07.403691 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b72f68d-584a-4213-b865-fc759eadd98e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:07.426594 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-9b72f68d-584a-4213-b865-fc759eadd98e None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:06:07.427974 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-9b72f68d-584a-4213-b865-fc759eadd98e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:06:07.497295 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de13671-fd7c-4f8e-a0f2-1262809638b6 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:07.506097 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 336/682] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:06:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 19:06:07.506582 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:07.508336 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b72f68d-584a-4213-b865-fc759eadd98e None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:07.510894 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:07.539701 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7732bd2-5ee8-4c12-8c91-a68c43d4692d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:07.539701 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7732bd2-5ee8-4c12-8c91-a68c43d4692d None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:07.539701 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7732bd2-5ee8-4c12-8c91-a68c43d4692d None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:07.539701 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7732bd2-5ee8-4c12-8c91-a68c43d4692d None admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:07.540137 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7732bd2-5ee8-4c12-8c91-a68c43d4692d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:07.540137 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7732bd2-5ee8-4c12-8c91-a68c43d4692d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:07.541208 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7732bd2-5ee8-4c12-8c91-a68c43d4692d None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:07.561675 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 347/683] 10.222.0.61 () {56 vars in 1148 bytes} [Mon Aug 22 19:06:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:07.567359 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7732bd2-5ee8-4c12-8c91-a68c43d4692d None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:07.664574 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b72f68d-584a-4213-b865-fc759eadd98e None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:07.701680 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7732bd2-5ee8-4c12-8c91-a68c43d4692d None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:07.707316 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33255628-496c-42b1-9e4a-51c59e6f1d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:07.708061 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33255628-496c-42b1-9e4a-51c59e6f1d95 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:07.708410 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33255628-496c-42b1-9e4a-51c59e6f1d95 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:07.708763 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33255628-496c-42b1-9e4a-51c59e6f1d95 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:07.720157 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33255628-496c-42b1-9e4a-51c59e6f1d95 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:07.721802 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33255628-496c-42b1-9e4a-51c59e6f1d95 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:07.736590 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-33255628-496c-42b1-9e4a-51c59e6f1d95 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 22 19:06:07.736590 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:07.736590 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:07.736590 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:07.736590 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:07.736590 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:07.736590 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:07.736590 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:07.736590 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:07.736590 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:07.736590 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:07.736590 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:07.736590 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 19:06:07.736590 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 19:06:07.736590 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 19:06:07.736590 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 19:06:07.737859 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:07.737859 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:07.737859 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 19:06:07.737859 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 19:06:07.737859 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 19:06:07.737859 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 19:06:07.737859 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:07.737859 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:07.737859 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:07.737859 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:07.737859 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:07.737859 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:07.737859 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:07.737859 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:07.737859 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:07.737859 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:07.737859 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:07.737859 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:07.739403 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:07.739403 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:07.739403 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 19:06:07.739403 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 19:06:07.739403 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 19:06:07.739403 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 19:06:07.739403 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 19:06:07.739403 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 19:06:07.739403 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 22 19:06:07.739403 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:06:07.741093 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 337/684] 10.222.0.61 () {56 vars in 1096 bytes} [Mon Aug 22 19:06:07 2022] GET /identity/v3/users/swift => generated 83 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:07.747636 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33255628-496c-42b1-9e4a-51c59e6f1d95 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:07.755154 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17bf573-6bd1-4650-ba88-1952270c8617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:07.756089 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17bf573-6bd1-4650-ba88-1952270c8617 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:07.756652 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17bf573-6bd1-4650-ba88-1952270c8617 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:07.757139 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17bf573-6bd1-4650-ba88-1952270c8617 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:07.758093 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17bf573-6bd1-4650-ba88-1952270c8617 None admin] RBAC: Adding query filter params (role.id=324bd04d55c34b61a5282c9f6ed73f46, scope.project.id=e07dc5ebdc504c6da706a2d21c0a3970, user.id=56e79498873d4ed29a0e19bbea9b9214) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:07.758714 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17bf573-6bd1-4650-ba88-1952270c8617 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:07.760403 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17bf573-6bd1-4650-ba88-1952270c8617 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:07.768256 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 348/685] 10.222.0.61 () {56 vars in 1374 bytes} [Mon Aug 22 19:06:07 2022] GET /identity/v3/role_assignments?user.id=56e79498873d4ed29a0e19bbea9b9214&scope.project.id=e07dc5ebdc504c6da706a2d21c0a3970&role.id=324bd04d55c34b61a5282c9f6ed73f46 => generated 608 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 19:06:07.818986 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17bf573-6bd1-4650-ba88-1952270c8617 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:07.841329 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:07.864260 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:07.880493 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33255628-496c-42b1-9e4a-51c59e6f1d95 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:07.927999 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4141cb-0795-4df0-9bb8-8de277953343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:07.928625 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4141cb-0795-4df0-9bb8-8de277953343 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:07.928903 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4141cb-0795-4df0-9bb8-8de277953343 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:07.929170 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4141cb-0795-4df0-9bb8-8de277953343 None admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:07.930486 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4141cb-0795-4df0-9bb8-8de277953343 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:07.930793 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4141cb-0795-4df0-9bb8-8de277953343 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:07.933333 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4141cb-0795-4df0-9bb8-8de277953343 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:07.966038 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 338/686] 10.222.0.61 () {56 vars in 1135 bytes} [Mon Aug 22 19:06:07 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:07.971269 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17bf573-6bd1-4650-ba88-1952270c8617 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:07.972487 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4141cb-0795-4df0-9bb8-8de277953343 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:08.026410 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce21a8e-b904-431b-8092-f171189cfca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:08.027051 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce21a8e-b904-431b-8092-f171189cfca5 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:08.027414 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce21a8e-b904-431b-8092-f171189cfca5 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:08.027792 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce21a8e-b904-431b-8092-f171189cfca5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:08.039738 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce21a8e-b904-431b-8092-f171189cfca5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 19:06:08.046762 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-cce21a8e-b904-431b-8092-f171189cfca5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 19:06:08.046762 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:08.046762 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:08.046762 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:08.046762 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:08.046762 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:08.046762 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:08.046762 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:08.046762 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:08.046762 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:08.046762 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:08.046762 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:08.046762 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 19:06:08.046762 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 19:06:08.046762 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 19:06:08.046762 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 19:06:08.047812 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 19:06:08.047812 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 19:06:08.047812 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 19:06:08.047812 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 19:06:08.047812 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:08.047812 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:08.047812 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:08.047812 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:08.047812 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:08.047812 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:08.047812 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:08.047812 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:08.047812 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:08.047812 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:08.047812 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:08.047812 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:08.048685 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:08.048685 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:08.048685 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:08.048685 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:08.048685 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 19:06:08.048685 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 19:06:08.048685 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 19:06:08.048685 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 19:06:08.048685 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 19:06:08.048685 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 19:06:08.048685 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 19:06:08.048685 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:06:08.052047 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 349/687] 10.222.0.61 () {56 vars in 1096 bytes} [Mon Aug 22 19:06:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 19:06:08.054238 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce21a8e-b904-431b-8092-f171189cfca5 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:08.101522 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4141cb-0795-4df0-9bb8-8de277953343 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:08.129229 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:08.149210 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a7dc29-f18e-495b-9a93-c31538485d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:08.150118 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a7dc29-f18e-495b-9a93-c31538485d07 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:08.150661 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a7dc29-f18e-495b-9a93-c31538485d07 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:08.151139 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a7dc29-f18e-495b-9a93-c31538485d07 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:08.169220 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:08.184885 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce21a8e-b904-431b-8092-f171189cfca5 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:08.230386 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79103309-e625-4df7-b2ff-d2bfc0fa29fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:08.230616 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79103309-e625-4df7-b2ff-d2bfc0fa29fc None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:08.230853 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79103309-e625-4df7-b2ff-d2bfc0fa29fc None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:08.231126 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79103309-e625-4df7-b2ff-d2bfc0fa29fc None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:08.231835 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79103309-e625-4df7-b2ff-d2bfc0fa29fc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:08.232138 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79103309-e625-4df7-b2ff-d2bfc0fa29fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:08.235186 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79103309-e625-4df7-b2ff-d2bfc0fa29fc None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:08.237465 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a7dc29-f18e-495b-9a93-c31538485d07 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:08.240430 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a7dc29-f18e-495b-9a93-c31538485d07 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:08.246003 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 350/688] 10.222.0.61 () {56 vars in 1099 bytes} [Mon Aug 22 19:06:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 19:06:08.252387 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79103309-e625-4df7-b2ff-d2bfc0fa29fc None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:08.377333 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79103309-e625-4df7-b2ff-d2bfc0fa29fc None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:08.402450 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 339/689] 10.222.0.61 () {58 vars in 1282 bytes} [Mon Aug 22 19:06:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 19:06:08.408236 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a7dc29-f18e-495b-9a93-c31538485d07 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:08.425638 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7f9004-e0dc-49b9-b1b1-66b46cfc7aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:08.426292 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7f9004-e0dc-49b9-b1b1-66b46cfc7aef None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:08.426570 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7f9004-e0dc-49b9-b1b1-66b46cfc7aef None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:08.426854 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7f9004-e0dc-49b9-b1b1-66b46cfc7aef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:08.442198 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7f9004-e0dc-49b9-b1b1-66b46cfc7aef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:08.443569 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7f9004-e0dc-49b9-b1b1-66b46cfc7aef None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:08.455961 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:08.461842 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-3f7f9004-e0dc-49b9-b1b1-66b46cfc7aef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:06:08.461842 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:08.461842 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:08.461842 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:08.461842 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:08.461842 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:08.461842 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:08.461842 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:08.461842 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:08.461842 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:08.461842 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:08.461842 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:08.461842 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 19:06:08.461842 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 19:06:08.461842 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 19:06:08.461842 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 19:06:08.463305 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:08.463305 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:08.463305 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:08.463305 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:08.463305 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:08.463305 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:08.463305 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:08.463305 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:08.463305 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:08.463305 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:08.463305 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:08.463305 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:08.463305 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:08.463305 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:08.463305 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:08.463305 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:08.463305 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 19:06:08.464328 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 19:06:08.464328 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:06:08.464328 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:06:08.466679 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 351/690] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:06:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:08.473002 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7f9004-e0dc-49b9-b1b1-66b46cfc7aef None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:08.487820 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:08.607018 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a7dc29-f18e-495b-9a93-c31538485d07 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:08.649715 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89cb469-5a25-429a-80de-7999e265426e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:08.650628 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89cb469-5a25-429a-80de-7999e265426e None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:08.651945 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89cb469-5a25-429a-80de-7999e265426e None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:08.652383 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89cb469-5a25-429a-80de-7999e265426e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:08.675835 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89cb469-5a25-429a-80de-7999e265426e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:08.678532 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89cb469-5a25-429a-80de-7999e265426e None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:08.686751 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7f9004-e0dc-49b9-b1b1-66b46cfc7aef None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:08.716736 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-b89cb469-5a25-429a-80de-7999e265426e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:06:08.716736 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:08.716736 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:08.716736 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:08.716736 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:08.716736 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:08.716736 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:08.716736 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:08.716736 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:08.716736 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:08.716736 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:08.716736 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:08.716736 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 19:06:08.716736 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 19:06:08.716736 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 19:06:08.716736 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 19:06:08.718007 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:08.718007 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:08.718007 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:08.718007 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:08.718007 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:08.718007 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:08.718007 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:08.718007 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:08.718007 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:08.718007 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:08.718007 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:08.718007 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:08.718007 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:08.718007 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:08.718007 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:08.718007 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:08.718007 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 19:06:08.719686 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 19:06:08.719686 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:06:08.719686 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:06:08.719686 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 340/691] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:06:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:08.728570 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89cb469-5a25-429a-80de-7999e265426e None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:08.736024 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1ad4a5-5779-4c46-b2e0-6253b8f55343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:08.736642 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1ad4a5-5779-4c46-b2e0-6253b8f55343 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:08.736918 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1ad4a5-5779-4c46-b2e0-6253b8f55343 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:08.737187 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1ad4a5-5779-4c46-b2e0-6253b8f55343 None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:08.737837 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1ad4a5-5779-4c46-b2e0-6253b8f55343 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:08.738141 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1ad4a5-5779-4c46-b2e0-6253b8f55343 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:08.741256 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1ad4a5-5779-4c46-b2e0-6253b8f55343 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:08.762854 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 352/692] 10.222.0.61 () {56 vars in 1109 bytes} [Mon Aug 22 19:06:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:08.770885 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1ad4a5-5779-4c46-b2e0-6253b8f55343 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:08.829960 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:08.869177 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89cb469-5a25-429a-80de-7999e265426e None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:08.912697 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6762e7-1d96-44f8-b8c4-45891f9235d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:08.913320 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6762e7-1d96-44f8-b8c4-45891f9235d4 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:08.913601 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6762e7-1d96-44f8-b8c4-45891f9235d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:08.913876 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6762e7-1d96-44f8-b8c4-45891f9235d4 None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:08.914802 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6762e7-1d96-44f8-b8c4-45891f9235d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:08.915106 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6762e7-1d96-44f8-b8c4-45891f9235d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:08.916058 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:08.917972 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6762e7-1d96-44f8-b8c4-45891f9235d4 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:08.936573 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 341/693] 10.222.0.61 () {56 vars in 1109 bytes} [Mon Aug 22 19:06:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:08.938275 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791cbc22-9495-4efa-862f-b650f0cd4a47 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:08.938560 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791cbc22-9495-4efa-862f-b650f0cd4a47 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:08.938834 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791cbc22-9495-4efa-862f-b650f0cd4a47 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:08.940683 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 342/694] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:06:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:06:08.944289 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791cbc22-9495-4efa-862f-b650f0cd4a47 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:08.965419 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1ad4a5-5779-4c46-b2e0-6253b8f55343 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:09.003935 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ec7177-74ba-41ee-9305-75db0cab7577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:09.005159 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ec7177-74ba-41ee-9305-75db0cab7577 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:09.005643 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ec7177-74ba-41ee-9305-75db0cab7577 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:09.006168 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ec7177-74ba-41ee-9305-75db0cab7577 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:09.014387 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ec7177-74ba-41ee-9305-75db0cab7577 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:09.016804 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ec7177-74ba-41ee-9305-75db0cab7577 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:09.029549 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-f8ec7177-74ba-41ee-9305-75db0cab7577 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 22 19:06:09.029549 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:09.029549 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:09.029549 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:09.029549 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:09.029549 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:09.029549 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:09.029549 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:09.029549 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:09.029549 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:09.029549 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:09.029549 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:09.029549 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 19:06:09.029549 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 19:06:09.029549 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 19:06:09.029549 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 19:06:09.030513 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:09.030513 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:09.030513 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 19:06:09.030513 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 19:06:09.030513 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 19:06:09.030513 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 19:06:09.030513 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:09.030513 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:09.030513 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:09.030513 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:09.030513 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:09.030513 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:09.030513 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:09.030513 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:09.030513 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:09.030513 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:09.030513 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:09.030513 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:09.032081 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:09.032081 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:09.032081 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 19:06:09.032081 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 19:06:09.032081 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 19:06:09.032081 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 19:06:09.032081 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 19:06:09.032081 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 19:06:09.032081 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 22 19:06:09.032081 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:06:09.039132 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 353/695] 10.222.0.61 () {56 vars in 1093 bytes} [Mon Aug 22 19:06:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:09.040789 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc11e1d1-f299-4930-a17f-e5a065cc50a1 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:09.041296 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc11e1d1-f299-4930-a17f-e5a065cc50a1 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:09.041771 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc11e1d1-f299-4930-a17f-e5a065cc50a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:09.061370 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-bc11e1d1-f299-4930-a17f-e5a065cc50a1 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:06:09.063045 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-bc11e1d1-f299-4930-a17f-e5a065cc50a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:06:09.070008 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791cbc22-9495-4efa-862f-b650f0cd4a47 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:09.103630 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:09.110215 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450fdeaa-4ee5-487b-aae4-92c0892ce3cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:09.111280 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450fdeaa-4ee5-487b-aae4-92c0892ce3cc None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:09.111826 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450fdeaa-4ee5-487b-aae4-92c0892ce3cc None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:09.112341 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450fdeaa-4ee5-487b-aae4-92c0892ce3cc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:09.120721 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450fdeaa-4ee5-487b-aae4-92c0892ce3cc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:09.122468 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450fdeaa-4ee5-487b-aae4-92c0892ce3cc None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:09.134156 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-450fdeaa-4ee5-487b-aae4-92c0892ce3cc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 19:06:09.134156 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:09.134156 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:09.134156 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:09.134156 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:09.134156 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:09.134156 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:09.134156 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:09.134156 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:09.134156 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:09.134156 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:09.134156 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:09.134156 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 19:06:09.134156 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 19:06:09.134156 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 19:06:09.135074 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 19:06:09.135074 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:09.135074 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:09.135074 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:09.135074 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:09.135074 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:09.135074 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:09.135074 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:09.135074 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:09.135074 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:09.135074 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:09.135074 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:09.135074 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:09.135074 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:09.135074 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:09.135074 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:09.135074 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:09.136020 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 19:06:09.136020 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 19:06:09.136020 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 19:06:09.136020 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 19:06:09.136020 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 19:06:09.136020 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 19:06:09.136020 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 19:06:09.136020 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:06:09.138361 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 343/696] 10.222.0.61 () {56 vars in 1111 bytes} [Mon Aug 22 19:06:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:09.139480 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450fdeaa-4ee5-487b-aae4-92c0892ce3cc None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:09.155309 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 354/697] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:06:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 19:06:09.160788 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc11e1d1-f299-4930-a17f-e5a065cc50a1 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:09.243447 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:09.299089 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc11e1d1-f299-4930-a17f-e5a065cc50a1 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:09.338565 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0274dfe1-a478-4337-9b80-25034977fa42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:09.339547 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0274dfe1-a478-4337-9b80-25034977fa42 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:09.340084 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0274dfe1-a478-4337-9b80-25034977fa42 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:09.340595 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0274dfe1-a478-4337-9b80-25034977fa42 None admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:09.341662 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0274dfe1-a478-4337-9b80-25034977fa42 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:09.342187 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0274dfe1-a478-4337-9b80-25034977fa42 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:09.343878 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0274dfe1-a478-4337-9b80-25034977fa42 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:09.357838 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 355/698] 10.222.0.61 () {56 vars in 1148 bytes} [Mon Aug 22 19:06:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:09.363677 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450fdeaa-4ee5-487b-aae4-92c0892ce3cc None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:09.366073 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0274dfe1-a478-4337-9b80-25034977fa42 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:09.429257 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbd2103-eeaa-40ba-998e-6cee8f020f03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:09.429872 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbd2103-eeaa-40ba-998e-6cee8f020f03 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:09.430168 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbd2103-eeaa-40ba-998e-6cee8f020f03 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:09.430424 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbd2103-eeaa-40ba-998e-6cee8f020f03 None admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:09.431163 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbd2103-eeaa-40ba-998e-6cee8f020f03 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:09.431469 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbd2103-eeaa-40ba-998e-6cee8f020f03 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:09.432921 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbd2103-eeaa-40ba-998e-6cee8f020f03 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:09.467316 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 344/699] 10.222.0.61 () {56 vars in 1133 bytes} [Mon Aug 22 19:06:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:09.473091 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbd2103-eeaa-40ba-998e-6cee8f020f03 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:09.488358 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0274dfe1-a478-4337-9b80-25034977fa42 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:09.508463 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:09.530565 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036b58ce-89ca-4655-9e78-93055a0ae05f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:09.531279 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036b58ce-89ca-4655-9e78-93055a0ae05f None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:09.531631 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036b58ce-89ca-4655-9e78-93055a0ae05f None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:09.532007 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036b58ce-89ca-4655-9e78-93055a0ae05f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:09.532804 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036b58ce-89ca-4655-9e78-93055a0ae05f None admin] RBAC: Adding query filter params (role.id=324bd04d55c34b61a5282c9f6ed73f46, scope.project.id=e07dc5ebdc504c6da706a2d21c0a3970, user.id=2824359491934d0dacaa21a718becee5) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:09.533187 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036b58ce-89ca-4655-9e78-93055a0ae05f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:09.534683 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036b58ce-89ca-4655-9e78-93055a0ae05f None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:09.541175 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 356/700] 10.222.0.61 () {56 vars in 1374 bytes} [Mon Aug 22 19:06:09 2022] GET /identity/v3/role_assignments?user.id=2824359491934d0dacaa21a718becee5&scope.project.id=e07dc5ebdc504c6da706a2d21c0a3970&role.id=324bd04d55c34b61a5282c9f6ed73f46 => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:09.542067 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036b58ce-89ca-4655-9e78-93055a0ae05f None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:09.548677 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:09.610177 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbd2103-eeaa-40ba-998e-6cee8f020f03 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:09.650189 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e83405-2499-49e6-ba61-456ec34a0ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:09.651217 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e83405-2499-49e6-ba61-456ec34a0ca7 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:09.651729 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e83405-2499-49e6-ba61-456ec34a0ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:09.652214 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e83405-2499-49e6-ba61-456ec34a0ca7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:09.659971 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036b58ce-89ca-4655-9e78-93055a0ae05f None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:09.729801 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba8955f-8651-41fa-b051-6521690d873a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:09.730457 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba8955f-8651-41fa-b051-6521690d873a None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:09.730736 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba8955f-8651-41fa-b051-6521690d873a None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:09.731008 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba8955f-8651-41fa-b051-6521690d873a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:09.732356 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e83405-2499-49e6-ba61-456ec34a0ca7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:09.734344 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e83405-2499-49e6-ba61-456ec34a0ca7 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:09.752450 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba8955f-8651-41fa-b051-6521690d873a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:09.753905 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba8955f-8651-41fa-b051-6521690d873a None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:09.769858 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-8ba8955f-8651-41fa-b051-6521690d873a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:06:09.769858 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:09.769858 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:09.769858 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:09.769858 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:09.769858 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:09.769858 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:09.769858 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:09.769858 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:09.769858 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:09.769858 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:09.769858 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:09.769858 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 19:06:09.769858 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 19:06:09.769858 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 19:06:09.769858 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 19:06:09.770947 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:09.770947 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:09.770947 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:09.770947 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:09.770947 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:09.770947 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:09.770947 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:09.770947 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:09.770947 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:09.770947 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:09.770947 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:09.770947 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:09.770947 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:09.770947 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:09.770947 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:09.770947 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:09.770947 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 19:06:09.771919 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 19:06:09.771919 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:06:09.771919 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:06:09.771919 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 357/701] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:06:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:09.779918 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba8955f-8651-41fa-b051-6521690d873a None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:09.814924 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:09.825416 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:09.889912 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 345/702] 10.222.0.61 () {58 vars in 1282 bytes} [Mon Aug 22 19:06:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 22 19:06:09.892104 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9f0786-b95c-4890-bcde-d7d072039707 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:09.892386 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9f0786-b95c-4890-bcde-d7d072039707 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:09.892657 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9f0786-b95c-4890-bcde-d7d072039707 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:09.894021 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 346/703] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:06:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:06:09.898449 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9f0786-b95c-4890-bcde-d7d072039707 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:09.921080 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba8955f-8651-41fa-b051-6521690d873a None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:09.977814 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594ec006-720a-4c66-a347-bd6ff84809e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:09.978429 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594ec006-720a-4c66-a347-bd6ff84809e7 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:09.978706 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594ec006-720a-4c66-a347-bd6ff84809e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:09.978973 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594ec006-720a-4c66-a347-bd6ff84809e7 None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:09.979627 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594ec006-720a-4c66-a347-bd6ff84809e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:09.979954 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594ec006-720a-4c66-a347-bd6ff84809e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:09.981355 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594ec006-720a-4c66-a347-bd6ff84809e7 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:10.002573 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 358/704] 10.222.0.61 () {56 vars in 1109 bytes} [Mon Aug 22 19:06:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 19:06:10.004885 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c8453d-b618-42b4-8b67-e502d0bfa146 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:10.005382 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c8453d-b618-42b4-8b67-e502d0bfa146 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:10.006408 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c8453d-b618-42b4-8b67-e502d0bfa146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:10.026984 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9f0786-b95c-4890-bcde-d7d072039707 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:10.031988 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-64c8453d-b618-42b4-8b67-e502d0bfa146 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:06:10.033423 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-64c8453d-b618-42b4-8b67-e502d0bfa146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:06:10.069117 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac6af5b-2a6f-4880-ab9e-b6bdc7b41cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:10.069734 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac6af5b-2a6f-4880-ab9e-b6bdc7b41cdc None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:10.070018 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac6af5b-2a6f-4880-ab9e-b6bdc7b41cdc None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:10.070290 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac6af5b-2a6f-4880-ab9e-b6bdc7b41cdc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:10.081023 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac6af5b-2a6f-4880-ab9e-b6bdc7b41cdc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:10.082486 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac6af5b-2a6f-4880-ab9e-b6bdc7b41cdc None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:10.098659 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-8ac6af5b-2a6f-4880-ab9e-b6bdc7b41cdc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:06:10.098659 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:10.098659 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:10.098659 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:10.098659 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:10.098659 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:10.098659 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:10.098659 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:10.098659 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:10.098659 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:10.098659 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:10.098659 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:10.098659 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 19:06:10.098659 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 19:06:10.098659 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 19:06:10.098659 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 19:06:10.099762 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:10.099762 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:10.099762 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:10.099762 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:10.099762 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:10.099762 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:10.099762 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:10.099762 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:10.099762 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:10.099762 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:10.099762 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:10.099762 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:10.099762 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:10.099762 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:10.099762 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:10.099762 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:10.099762 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 19:06:10.100762 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 19:06:10.100762 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:06:10.100762 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:06:10.100762 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 347/705] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:06:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 19:06:10.101005 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac6af5b-2a6f-4880-ab9e-b6bdc7b41cdc None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:10.104118 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:10.138134 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 359/706] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:06:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 19:06:10.139500 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c8453d-b618-42b4-8b67-e502d0bfa146 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:10.170609 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:10.240238 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac6af5b-2a6f-4880-ab9e-b6bdc7b41cdc None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:10.278669 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c366dd00-92e1-4073-a6f5-d04ebce8caa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:10.279528 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c366dd00-92e1-4073-a6f5-d04ebce8caa4 None admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:10.280102 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c366dd00-92e1-4073-a6f5-d04ebce8caa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:10.281243 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c366dd00-92e1-4073-a6f5-d04ebce8caa4 None admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:10.282168 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c366dd00-92e1-4073-a6f5-d04ebce8caa4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:10.284034 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c366dd00-92e1-4073-a6f5-d04ebce8caa4 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:10.318611 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 348/707] 10.222.0.61 () {60 vars in 1132 bytes} [Mon Aug 22 19:06:10 2022] POST /identity/v3/users => generated 254 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:06:10.327051 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c8453d-b618-42b4-8b67-e502d0bfa146 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:10.359988 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0bd47d-d541-4466-963c-6ec7a3a8679d None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:10.360602 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0bd47d-d541-4466-963c-6ec7a3a8679d None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:10.361039 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0bd47d-d541-4466-963c-6ec7a3a8679d None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:10.363073 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 349/708] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:06:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:06:10.369370 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5423a927-c297-42d5-beba-d7d191d81450 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:10.369650 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5423a927-c297-42d5-beba-d7d191d81450 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:10.369933 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5423a927-c297-42d5-beba-d7d191d81450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:10.398258 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff4b292-d89f-4c4b-9ed5-fefd33e5d2e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:10.400663 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff4b292-d89f-4c4b-9ed5-fefd33e5d2e4 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:10.401472 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff4b292-d89f-4c4b-9ed5-fefd33e5d2e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:10.402759 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff4b292-d89f-4c4b-9ed5-fefd33e5d2e4 None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:10.403694 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff4b292-d89f-4c4b-9ed5-fefd33e5d2e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:10.404853 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff4b292-d89f-4c4b-9ed5-fefd33e5d2e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:10.410046 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff4b292-d89f-4c4b-9ed5-fefd33e5d2e4 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:10.411270 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-5423a927-c297-42d5-beba-d7d191d81450 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:06:10.413193 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-5423a927-c297-42d5-beba-d7d191d81450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:06:10.434125 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 360/709] 10.222.0.61 () {56 vars in 1109 bytes} [Mon Aug 22 19:06:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:10.439624 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff4b292-d89f-4c4b-9ed5-fefd33e5d2e4 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:10.488106 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:10.497170 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 350/710] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:06:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 19:06:10.505940 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5423a927-c297-42d5-beba-d7d191d81450 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:10.585672 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:10.607845 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff4b292-d89f-4c4b-9ed5-fefd33e5d2e4 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:10.620829 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5423a927-c297-42d5-beba-d7d191d81450 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:10.644306 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27b7c47-d15b-49a4-b4c7-898b6d58855f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:10.645012 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27b7c47-d15b-49a4-b4c7-898b6d58855f None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:10.645285 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27b7c47-d15b-49a4-b4c7-898b6d58855f None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:10.645617 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27b7c47-d15b-49a4-b4c7-898b6d58855f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:10.653393 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27b7c47-d15b-49a4-b4c7-898b6d58855f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:10.654226 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27b7c47-d15b-49a4-b4c7-898b6d58855f None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:10.662268 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4cd1d3-aad5-4e53-a4aa-7169d64525b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:10.662911 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4cd1d3-aad5-4e53-a4aa-7169d64525b0 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:10.663191 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4cd1d3-aad5-4e53-a4aa-7169d64525b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:10.663464 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4cd1d3-aad5-4e53-a4aa-7169d64525b0 None admin] PATH_INFO: `/v3/services/network` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:10.664094 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4cd1d3-aad5-4e53-a4aa-7169d64525b0 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:10.667057 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4cd1d3-aad5-4e53-a4aa-7169d64525b0 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:10.674894 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-d27b7c47-d15b-49a4-b4c7-898b6d58855f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 19:06:10.674894 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:10.674894 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:10.674894 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:10.674894 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:10.674894 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:10.674894 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:10.674894 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:10.674894 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:10.674894 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:10.674894 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:10.674894 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:10.674894 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 19:06:10.674894 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 19:06:10.674894 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 19:06:10.675881 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 19:06:10.675881 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:10.675881 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:10.675881 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:10.675881 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:10.675881 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:10.675881 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:10.675881 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:10.675881 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:10.675881 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:10.675881 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:10.675881 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:10.675881 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:10.675881 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:10.675881 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:10.675881 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:10.675881 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:10.676811 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 19:06:10.676811 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 19:06:10.676811 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 19:06:10.676811 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 19:06:10.676811 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 19:06:10.676811 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 19:06:10.676811 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 19:06:10.676811 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:06:10.678886 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 361/711] 10.222.0.61 () {56 vars in 1111 bytes} [Mon Aug 22 19:06:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:10.681021 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-8a4cd1d3-aad5-4e53-a4aa-7169d64525b0 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 22 19:06:10.681021 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:10.681021 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 19:06:10.681021 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 19:06:10.681021 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 19:06:10.681021 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 19:06:10.681021 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 19:06:10.681021 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 19:06:10.681021 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 22 19:06:10.681021 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:06:10.681021 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 19:06:10.681021 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:06:10.681021 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:10.681021 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:10.681021 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:10.681021 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:10.681021 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:10.682071 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:10.682071 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:10.682071 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:10.682071 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:10.682071 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:10.682071 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:10.682071 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 19:06:10.682071 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 19:06:10.682071 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 19:06:10.682071 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 19:06:10.682071 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:10.682071 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:10.682071 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:10.682071 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:10.682071 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:10.682071 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:10.682921 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:10.682921 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:10.682921 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:10.682921 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:10.682921 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:10.682921 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:10.682921 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:10.682921 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:10.682921 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:10.682921 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:10.682921 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 19:06:10.682921 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 19:06:10.682921 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 22 19:06:10.682921 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:06:10.682921 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 351/712] 10.222.0.61 () {56 vars in 1111 bytes} [Mon Aug 22 19:06:10 2022] GET /identity/v3/services/network => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:10.688106 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4cd1d3-aad5-4e53-a4aa-7169d64525b0 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:10.688305 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27b7c47-d15b-49a4-b4c7-898b6d58855f None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:10.769598 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:10.834810 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27b7c47-d15b-49a4-b4c7-898b6d58855f None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:10.842272 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4cd1d3-aad5-4e53-a4aa-7169d64525b0 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:10.878790 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08fb944-7c56-41b7-8a5a-6fd0dae86ef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:10.879417 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08fb944-7c56-41b7-8a5a-6fd0dae86ef7 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:10.879734 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08fb944-7c56-41b7-8a5a-6fd0dae86ef7 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:10.880009 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08fb944-7c56-41b7-8a5a-6fd0dae86ef7 None admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:10.880834 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08fb944-7c56-41b7-8a5a-6fd0dae86ef7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:10.881143 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08fb944-7c56-41b7-8a5a-6fd0dae86ef7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:10.882716 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08fb944-7c56-41b7-8a5a-6fd0dae86ef7 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:10.887190 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c13bf3a-159a-418a-b9cf-2fc5c2699e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:10.888076 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c13bf3a-159a-418a-b9cf-2fc5c2699e73 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:10.888536 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c13bf3a-159a-418a-b9cf-2fc5c2699e73 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:10.888981 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c13bf3a-159a-418a-b9cf-2fc5c2699e73 None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:10.889855 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c13bf3a-159a-418a-b9cf-2fc5c2699e73 None admin] RBAC: Adding query filter params (name=network) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:10.890999 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c13bf3a-159a-418a-b9cf-2fc5c2699e73 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:10.892584 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c13bf3a-159a-418a-b9cf-2fc5c2699e73 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:10.893004 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:10.897823 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 362/713] 10.222.0.61 () {56 vars in 1148 bytes} [Mon Aug 22 19:06:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 19:06:10.898906 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 352/714] 10.222.0.61 () {56 vars in 1112 bytes} [Mon Aug 22 19:06:10 2022] GET /identity/v3/services?name=network => generated 124 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:10.899017 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08fb944-7c56-41b7-8a5a-6fd0dae86ef7 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:10.904347 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c13bf3a-159a-418a-b9cf-2fc5c2699e73 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:11.009264 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08fb944-7c56-41b7-8a5a-6fd0dae86ef7 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:11.033074 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c13bf3a-159a-418a-b9cf-2fc5c2699e73 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:11.050087 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:11.054232 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799ad5bc-200a-42ea-9575-0ca98bab01d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:11.054867 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799ad5bc-200a-42ea-9575-0ca98bab01d5 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:11.055145 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799ad5bc-200a-42ea-9575-0ca98bab01d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:11.055514 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799ad5bc-200a-42ea-9575-0ca98bab01d5 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:11.056096 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799ad5bc-200a-42ea-9575-0ca98bab01d5 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:11.057889 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799ad5bc-200a-42ea-9575-0ca98bab01d5 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:11.071750 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-799ad5bc-200a-42ea-9575-0ca98bab01d5 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 22 19:06:11.071750 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:11.071750 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 19:06:11.071750 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 19:06:11.071750 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 19:06:11.071750 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 19:06:11.071750 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 19:06:11.071750 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 19:06:11.071750 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 22 19:06:11.071750 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:06:11.071750 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 19:06:11.071750 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:06:11.071750 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:11.071750 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:11.071750 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:11.071750 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:11.071750 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:11.072778 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:11.072778 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:11.072778 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:11.072778 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:11.072778 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:11.072778 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:11.072778 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 19:06:11.072778 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 19:06:11.072778 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 19:06:11.072778 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 19:06:11.072778 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:11.072778 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:11.072778 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:11.072778 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:11.072778 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:11.072778 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:11.074252 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:11.074252 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:11.074252 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:11.074252 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:11.074252 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:11.074252 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:11.074252 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:11.074252 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:11.074252 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:11.074252 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:11.074252 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 19:06:11.074252 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 19:06:11.074252 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 22 19:06:11.074252 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:06:11.074252 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 363/715] 10.222.0.61 () {56 vars in 1129 bytes} [Mon Aug 22 19:06:10 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:11.075069 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799ad5bc-200a-42ea-9575-0ca98bab01d5 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:11.076310 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0fe15f-5430-4c2f-9e6a-1c70b5e6b039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:11.077210 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0fe15f-5430-4c2f-9e6a-1c70b5e6b039 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:11.078412 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0fe15f-5430-4c2f-9e6a-1c70b5e6b039 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:11.078875 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0fe15f-5430-4c2f-9e6a-1c70b5e6b039 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:11.079821 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0fe15f-5430-4c2f-9e6a-1c70b5e6b039 None admin] RBAC: Adding query filter params (role.id=163c30c3b3c246beb849ce176f02d9da, scope.project.id=e07dc5ebdc504c6da706a2d21c0a3970, user.id=e7f43e2756b746a6825ca1a06b7d7c91) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:11.080350 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0fe15f-5430-4c2f-9e6a-1c70b5e6b039 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:11.081978 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0fe15f-5430-4c2f-9e6a-1c70b5e6b039 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:11.094804 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 353/716] 10.222.0.61 () {56 vars in 1374 bytes} [Mon Aug 22 19:06:10 2022] GET /identity/v3/role_assignments?user.id=e7f43e2756b746a6825ca1a06b7d7c91&scope.project.id=e07dc5ebdc504c6da706a2d21c0a3970&role.id=163c30c3b3c246beb849ce176f02d9da => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:11.096194 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0fe15f-5430-4c2f-9e6a-1c70b5e6b039 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:11.168882 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:11.238013 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799ad5bc-200a-42ea-9575-0ca98bab01d5 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:11.251216 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0fe15f-5430-4c2f-9e6a-1c70b5e6b039 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:11.291440 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cea4d04-1e30-4461-8f7f-12e76f2e5697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:11.292402 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cea4d04-1e30-4461-8f7f-12e76f2e5697 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:11.293125 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cea4d04-1e30-4461-8f7f-12e76f2e5697 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:11.293551 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cea4d04-1e30-4461-8f7f-12e76f2e5697 None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:11.294616 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cea4d04-1e30-4461-8f7f-12e76f2e5697 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:11.295071 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cea4d04-1e30-4461-8f7f-12e76f2e5697 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:11.297169 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cea4d04-1e30-4461-8f7f-12e76f2e5697 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:11.303280 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 354/717] 10.222.0.61 () {56 vars in 1124 bytes} [Mon Aug 22 19:06:11 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:11.310616 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cea4d04-1e30-4461-8f7f-12e76f2e5697 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:11.321249 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453421c2-c024-4922-8ac0-1ef8cc886583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:11.321876 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453421c2-c024-4922-8ac0-1ef8cc886583 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:11.322845 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453421c2-c024-4922-8ac0-1ef8cc886583 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:11.322845 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453421c2-c024-4922-8ac0-1ef8cc886583 None admin] PATH_INFO: `/v3/services` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:11.323073 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453421c2-c024-4922-8ac0-1ef8cc886583 None admin] RBAC: Adding query filter params (type=network) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:11.323379 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453421c2-c024-4922-8ac0-1ef8cc886583 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:11.325454 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453421c2-c024-4922-8ac0-1ef8cc886583 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:11.331977 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 364/718] 10.222.0.61 () {56 vars in 1112 bytes} [Mon Aug 22 19:06:11 2022] GET /identity/v3/services?type=network => generated 124 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:11.381176 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:11.448192 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cea4d04-1e30-4461-8f7f-12e76f2e5697 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:11.497505 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05acc428-bf7b-4d2c-9e61-b440497ff2c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:11.498313 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05acc428-bf7b-4d2c-9e61-b440497ff2c8 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:11.498499 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05acc428-bf7b-4d2c-9e61-b440497ff2c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:11.498761 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05acc428-bf7b-4d2c-9e61-b440497ff2c8 None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:11.499464 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05acc428-bf7b-4d2c-9e61-b440497ff2c8 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:11.499808 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05acc428-bf7b-4d2c-9e61-b440497ff2c8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:11.501220 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05acc428-bf7b-4d2c-9e61-b440497ff2c8 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:11.508117 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 355/719] 10.222.0.61 () {56 vars in 1124 bytes} [Mon Aug 22 19:06:11 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 19:06:11.779228 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c231f71-581c-4b8c-bfbb-a78086530bba None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:11.779570 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c231f71-581c-4b8c-bfbb-a78086530bba None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:11.779717 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c231f71-581c-4b8c-bfbb-a78086530bba None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:11.781924 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 365/720] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:06:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:06:11.790630 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a3fe66-3a49-4059-ae93-416be5f09d9b None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:11.791173 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a3fe66-3a49-4059-ae93-416be5f09d9b None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:11.791682 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a3fe66-3a49-4059-ae93-416be5f09d9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:11.820159 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-d9a3fe66-3a49-4059-ae93-416be5f09d9b None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:06:11.821834 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-d9a3fe66-3a49-4059-ae93-416be5f09d9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:06:11.929301 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 356/721] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:06:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 19:06:12.257346 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c231f71-581c-4b8c-bfbb-a78086530bba None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:12.388297 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c231f71-581c-4b8c-bfbb-a78086530bba None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:12.429531 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:12.439016 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711b5898-515c-430f-812a-fdc6f52d61c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:12.439667 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711b5898-515c-430f-812a-fdc6f52d61c7 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:12.439960 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711b5898-515c-430f-812a-fdc6f52d61c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:12.440237 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711b5898-515c-430f-812a-fdc6f52d61c7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:12.454752 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711b5898-515c-430f-812a-fdc6f52d61c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 19:06:12.460062 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-711b5898-515c-430f-812a-fdc6f52d61c7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 19:06:12.460062 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:12.460062 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:12.460062 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:12.460062 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:12.460062 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:12.460062 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:12.460062 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:12.460062 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:12.460062 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:12.460062 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:12.460062 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:12.460062 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 19:06:12.460062 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 19:06:12.460062 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 19:06:12.460062 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 19:06:12.461010 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 19:06:12.461010 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 19:06:12.461010 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 19:06:12.461010 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 19:06:12.461010 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:12.461010 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:12.461010 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:12.461010 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:12.461010 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:12.461010 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:12.461010 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:12.461010 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:12.461010 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:12.461010 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:12.461010 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:12.461010 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:12.461847 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:12.461847 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:12.461847 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:12.461847 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:12.461847 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 19:06:12.461847 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 19:06:12.461847 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 19:06:12.461847 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 19:06:12.461847 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 19:06:12.461847 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 19:06:12.461847 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 19:06:12.461847 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:06:12.461847 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 366/722] 10.222.0.61 () {56 vars in 1096 bytes} [Mon Aug 22 19:06:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:12.467069 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a3fe66-3a49-4059-ae93-416be5f09d9b None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:12.544582 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:12.628245 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a3fe66-3a49-4059-ae93-416be5f09d9b None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:12.681328 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e731c358-e4ad-4268-b6f9-9c00f8e54748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:12.681854 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e731c358-e4ad-4268-b6f9-9c00f8e54748 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:12.682133 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e731c358-e4ad-4268-b6f9-9c00f8e54748 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:12.682409 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e731c358-e4ad-4268-b6f9-9c00f8e54748 None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:12.683080 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e731c358-e4ad-4268-b6f9-9c00f8e54748 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:12.683379 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e731c358-e4ad-4268-b6f9-9c00f8e54748 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:12.684780 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e731c358-e4ad-4268-b6f9-9c00f8e54748 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:12.699504 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 357/723] 10.222.0.61 () {56 vars in 1099 bytes} [Mon Aug 22 19:06:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:12.705561 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711b5898-515c-430f-812a-fdc6f52d61c7 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:12.837482 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fce2de-4215-4c41-9965-a69ee10e3285 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:12.837837 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fce2de-4215-4c41-9965-a69ee10e3285 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:12.838037 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fce2de-4215-4c41-9965-a69ee10e3285 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:12.839655 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 358/724] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:06:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:06:12.847783 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2b443a-f75f-4ac9-a177-a9ab436c311d None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:12.848066 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2b443a-f75f-4ac9-a177-a9ab436c311d None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:12.848333 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2b443a-f75f-4ac9-a177-a9ab436c311d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:12.862764 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711b5898-515c-430f-812a-fdc6f52d61c7 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:12.872462 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-3c2b443a-f75f-4ac9-a177-a9ab436c311d None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:06:12.873691 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-3c2b443a-f75f-4ac9-a177-a9ab436c311d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:06:12.922023 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9b8fe1-22c8-40e8-86b9-748af9ef64fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:12.922671 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9b8fe1-22c8-40e8-86b9-748af9ef64fc None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:12.922946 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9b8fe1-22c8-40e8-86b9-748af9ef64fc None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:12.923226 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9b8fe1-22c8-40e8-86b9-748af9ef64fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:12.938723 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9b8fe1-22c8-40e8-86b9-748af9ef64fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:12.940151 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9b8fe1-22c8-40e8-86b9-748af9ef64fc None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:12.957043 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-df9b8fe1-22c8-40e8-86b9-748af9ef64fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:06:12.957043 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:12.957043 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:12.957043 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:12.957043 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:12.957043 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:12.957043 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:12.957043 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:12.957043 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:12.957043 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:12.957043 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:12.957043 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:12.957043 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 19:06:12.957043 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 19:06:12.957043 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 19:06:12.957043 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 19:06:12.958060 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:12.958060 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:12.958060 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:12.958060 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:12.958060 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:12.958060 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:12.958060 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:12.958060 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:12.958060 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:12.958060 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:12.958060 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:12.958060 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:12.958060 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:12.958060 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:12.958060 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:12.958060 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:12.958060 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 19:06:12.958960 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 19:06:12.958960 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:06:12.958960 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:06:12.958960 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 367/725] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:06:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:12.969058 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9b8fe1-22c8-40e8-86b9-748af9ef64fc None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:12.988455 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 359/726] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:06:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 22 19:06:13.039911 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:13.121031 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9b8fe1-22c8-40e8-86b9-748af9ef64fc None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:13.174154 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b6dfb5-a118-43f2-a748-3d087c2e2163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:13.174782 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b6dfb5-a118-43f2-a748-3d087c2e2163 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:13.175061 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b6dfb5-a118-43f2-a748-3d087c2e2163 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:13.175337 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b6dfb5-a118-43f2-a748-3d087c2e2163 None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:13.176016 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b6dfb5-a118-43f2-a748-3d087c2e2163 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:13.176322 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b6dfb5-a118-43f2-a748-3d087c2e2163 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:13.177686 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b6dfb5-a118-43f2-a748-3d087c2e2163 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:13.197198 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 368/727] 10.222.0.61 () {56 vars in 1109 bytes} [Mon Aug 22 19:06:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:13.202326 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2b443a-f75f-4ac9-a177-a9ab436c311d None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:13.238907 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:13.327152 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2b443a-f75f-4ac9-a177-a9ab436c311d None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:13.356657 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b6dfb5-a118-43f2-a748-3d087c2e2163 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:13.386277 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dda31e0-f733-4a8b-9a7b-1d348ac8f422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:13.387275 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dda31e0-f733-4a8b-9a7b-1d348ac8f422 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:13.387970 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dda31e0-f733-4a8b-9a7b-1d348ac8f422 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:13.388440 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dda31e0-f733-4a8b-9a7b-1d348ac8f422 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:13.400707 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dda31e0-f733-4a8b-9a7b-1d348ac8f422 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:13.403113 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dda31e0-f733-4a8b-9a7b-1d348ac8f422 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:13.427662 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-2dda31e0-f733-4a8b-9a7b-1d348ac8f422 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 22 19:06:13.427662 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:13.427662 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:13.427662 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:13.427662 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:13.427662 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:13.427662 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:13.427662 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:13.427662 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:13.427662 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:13.427662 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:13.427662 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:13.427662 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 19:06:13.427662 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 19:06:13.427662 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 19:06:13.427662 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 19:06:13.428651 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:13.428651 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:13.428651 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 19:06:13.428651 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 19:06:13.428651 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 19:06:13.428651 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 19:06:13.428651 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:13.428651 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:13.428651 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:13.428651 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:13.428651 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:13.428651 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:13.428651 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:13.428651 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:13.428651 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:13.428651 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:13.428651 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:13.428651 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:13.429595 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:13.429595 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:13.429595 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 19:06:13.429595 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 19:06:13.429595 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 19:06:13.429595 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 19:06:13.429595 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 19:06:13.429595 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 19:06:13.429595 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 22 19:06:13.429595 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:06:13.432968 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 360/728] 10.222.0.61 () {56 vars in 1096 bytes} [Mon Aug 22 19:06:13 2022] GET /identity/v3/users/swift => generated 83 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:13.439217 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dda31e0-f733-4a8b-9a7b-1d348ac8f422 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:13.525876 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:13.531592 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b6dfb5-a118-43f2-a748-3d087c2e2163 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:13.554075 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:13.567192 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dda31e0-f733-4a8b-9a7b-1d348ac8f422 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:13.599747 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2af738-c2e4-41cd-9d00-43e23473c82d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:13.600402 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2af738-c2e4-41cd-9d00-43e23473c82d None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:13.600681 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2af738-c2e4-41cd-9d00-43e23473c82d None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:13.600951 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2af738-c2e4-41cd-9d00-43e23473c82d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:13.604746 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058b68e7-d487-41f2-bfab-9113167c7bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:13.605678 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058b68e7-d487-41f2-bfab-9113167c7bb9 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:13.606164 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058b68e7-d487-41f2-bfab-9113167c7bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:13.606643 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058b68e7-d487-41f2-bfab-9113167c7bb9 None admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:13.607613 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058b68e7-d487-41f2-bfab-9113167c7bb9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:13.608146 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058b68e7-d487-41f2-bfab-9113167c7bb9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:13.610412 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058b68e7-d487-41f2-bfab-9113167c7bb9 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:13.630039 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 361/729] 10.222.0.61 () {56 vars in 1135 bytes} [Mon Aug 22 19:06:13 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 19:06:13.630744 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2af738-c2e4-41cd-9d00-43e23473c82d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 19:06:13.640029 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-da2af738-c2e4-41cd-9d00-43e23473c82d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 19:06:13.640029 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:13.640029 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:13.640029 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:13.640029 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:13.640029 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:13.640029 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:13.640029 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:13.640029 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:13.640029 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:13.640029 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:13.640029 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:13.640029 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 19:06:13.640029 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 19:06:13.640029 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 19:06:13.640029 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 19:06:13.641003 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 19:06:13.641003 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 19:06:13.641003 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 19:06:13.641003 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 19:06:13.641003 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:13.641003 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:13.641003 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:13.641003 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:13.641003 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:13.641003 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:13.641003 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:13.641003 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:13.641003 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:13.641003 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:13.641003 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:13.641003 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:13.641876 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:13.641876 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:13.641876 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:13.641876 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:13.641876 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 19:06:13.641876 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 19:06:13.641876 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 19:06:13.641876 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 19:06:13.641876 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 19:06:13.641876 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 19:06:13.641876 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 19:06:13.641876 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:06:13.641876 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 369/730] 10.222.0.61 () {56 vars in 1102 bytes} [Mon Aug 22 19:06:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 19:06:13.643736 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058b68e7-d487-41f2-bfab-9113167c7bb9 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:13.644333 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2af738-c2e4-41cd-9d00-43e23473c82d None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:13.764964 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2af738-c2e4-41cd-9d00-43e23473c82d None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:13.765957 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058b68e7-d487-41f2-bfab-9113167c7bb9 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:13.800700 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9009d6fd-f798-4e87-9538-9118ab0596dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:13.801233 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9009d6fd-f798-4e87-9538-9118ab0596dc None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:13.801514 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9009d6fd-f798-4e87-9538-9118ab0596dc None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:13.801786 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9009d6fd-f798-4e87-9538-9118ab0596dc None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:13.802354 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d57cd3d-88dc-4560-9aa6-4e29bd3744c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:13.802468 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9009d6fd-f798-4e87-9538-9118ab0596dc None admin] RBAC: Adding query filter params (name=service) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:13.802758 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9009d6fd-f798-4e87-9538-9118ab0596dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:13.802958 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d57cd3d-88dc-4560-9aa6-4e29bd3744c0 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:13.803232 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d57cd3d-88dc-4560-9aa6-4e29bd3744c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:13.803498 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d57cd3d-88dc-4560-9aa6-4e29bd3744c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:13.804149 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9009d6fd-f798-4e87-9538-9118ab0596dc None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:13.816489 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 370/731] 10.222.0.61 () {56 vars in 1103 bytes} [Mon Aug 22 19:06:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:13.816489 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fc04dc-f89b-42e5-947f-f4bbf2df8fc5 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:13.816489 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fc04dc-f89b-42e5-947f-f4bbf2df8fc5 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:13.816489 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fc04dc-f89b-42e5-947f-f4bbf2df8fc5 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:13.816489 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 371/732] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:06:13 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:06:13.816489 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39701ad9-b64a-4aeb-9da4-b628234e4336 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:13.816992 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39701ad9-b64a-4aeb-9da4-b628234e4336 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:13.817385 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39701ad9-b64a-4aeb-9da4-b628234e4336 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:13.818424 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 372/733] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:06:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:06:13.819005 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39701ad9-b64a-4aeb-9da4-b628234e4336 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:13.830933 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:13.847427 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:13.889585 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d57cd3d-88dc-4560-9aa6-4e29bd3744c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:13.891054 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d57cd3d-88dc-4560-9aa6-4e29bd3744c0 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:13.932723 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39701ad9-b64a-4aeb-9da4-b628234e4336 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:13.967929 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917ce8ba-68f4-48d4-a463-d6f5ebced4ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:13.969103 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917ce8ba-68f4-48d4-a463-d6f5ebced4ec None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:13.969380 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917ce8ba-68f4-48d4-a463-d6f5ebced4ec None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:13.969647 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917ce8ba-68f4-48d4-a463-d6f5ebced4ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:13.982008 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917ce8ba-68f4-48d4-a463-d6f5ebced4ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:13.983452 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917ce8ba-68f4-48d4-a463-d6f5ebced4ec None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:13.995223 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 362/734] 10.222.0.61 () {58 vars in 1282 bytes} [Mon Aug 22 19:06:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 19:06:13.997001 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce2ab67-d1de-49c3-9222-cdcd4d498bad None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:13.997559 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce2ab67-d1de-49c3-9222-cdcd4d498bad None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:13.998062 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce2ab67-d1de-49c3-9222-cdcd4d498bad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:13.998381 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-917ce8ba-68f4-48d4-a463-d6f5ebced4ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:06:13.998381 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:13.998381 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:13.998381 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:13.998381 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:13.998381 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:13.998381 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:13.998381 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:13.998381 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:13.998381 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:13.998381 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:13.998381 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:13.998381 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 19:06:13.998381 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 19:06:13.998381 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 19:06:13.998381 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 19:06:13.999341 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:13.999341 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:13.999341 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:13.999341 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:13.999341 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:13.999341 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:13.999341 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:13.999341 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:13.999341 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:13.999341 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:13.999341 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:13.999341 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:13.999341 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:13.999341 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:13.999341 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:13.999341 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:13.999341 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 19:06:14.000373 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 19:06:14.000373 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:06:14.000373 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:06:14.000373 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 373/735] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:06:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:14.001199 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67234fd-de25-45c7-b7d3-e48395f9d057 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:14.001199 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67234fd-de25-45c7-b7d3-e48395f9d057 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:14.001199 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67234fd-de25-45c7-b7d3-e48395f9d057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:14.019217 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-b67234fd-de25-45c7-b7d3-e48395f9d057 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:06:14.020431 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-b67234fd-de25-45c7-b7d3-e48395f9d057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:06:14.023089 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-8ce2ab67-d1de-49c3-9222-cdcd4d498bad None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:06:14.024905 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-8ce2ab67-d1de-49c3-9222-cdcd4d498bad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:06:14.072626 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:14.092377 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 374/736] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:06:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 19:06:14.093909 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67234fd-de25-45c7-b7d3-e48395f9d057 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:14.097680 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:14.114850 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 363/737] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:06:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 19:06:14.114850 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce2ab67-d1de-49c3-9222-cdcd4d498bad None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:14.208160 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67234fd-de25-45c7-b7d3-e48395f9d057 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:14.232643 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce2ab67-d1de-49c3-9222-cdcd4d498bad None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:14.256192 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707c4f5d-ce77-44b2-b5c7-06263a1a770d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:14.257114 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707c4f5d-ce77-44b2-b5c7-06263a1a770d None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:14.257518 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707c4f5d-ce77-44b2-b5c7-06263a1a770d None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:14.258206 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707c4f5d-ce77-44b2-b5c7-06263a1a770d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:14.269042 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40be01ee-035d-47e7-a399-1dde2972d185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:14.269648 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40be01ee-035d-47e7-a399-1dde2972d185 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:14.269924 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40be01ee-035d-47e7-a399-1dde2972d185 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:14.270194 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40be01ee-035d-47e7-a399-1dde2972d185 None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:14.270837 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40be01ee-035d-47e7-a399-1dde2972d185 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:14.271107 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707c4f5d-ce77-44b2-b5c7-06263a1a770d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:14.271223 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40be01ee-035d-47e7-a399-1dde2972d185 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:14.272504 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40be01ee-035d-47e7-a399-1dde2972d185 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:14.272613 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707c4f5d-ce77-44b2-b5c7-06263a1a770d None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:14.289113 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 364/738] 10.222.0.61 () {56 vars in 1109 bytes} [Mon Aug 22 19:06:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:14.289943 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-707c4f5d-ce77-44b2-b5c7-06263a1a770d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:06:14.289943 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:14.289943 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:14.289943 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:14.289943 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:14.289943 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:14.289943 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:14.289943 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:14.289943 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:14.289943 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:14.289943 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:14.289943 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:14.289943 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 19:06:14.289943 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 19:06:14.289943 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 19:06:14.289943 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 19:06:14.290870 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:14.290870 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:14.290870 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:14.290870 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:14.290870 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:14.290870 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:14.290870 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:14.290870 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:14.290870 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:14.290870 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:14.290870 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:14.290870 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:14.290870 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:14.290870 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:14.290870 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:14.290870 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:14.290870 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 19:06:14.292205 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 19:06:14.292205 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:06:14.292205 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:06:14.292583 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1692d96-31c3-4ead-9d8e-fcbf45be071d None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:14.292583 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1692d96-31c3-4ead-9d8e-fcbf45be071d None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:14.292583 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1692d96-31c3-4ead-9d8e-fcbf45be071d None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:14.292973 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 375/739] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:06:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:14.296703 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 365/740] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:06:14 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:06:14.298089 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707c4f5d-ce77-44b2-b5c7-06263a1a770d None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:14.298223 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1692d96-31c3-4ead-9d8e-fcbf45be071d None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:14.341692 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:14.367533 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:14.412767 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707c4f5d-ce77-44b2-b5c7-06263a1a770d None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:14.416742 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1692d96-31c3-4ead-9d8e-fcbf45be071d None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:14.447560 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2bdc25-4206-48f2-98b4-c5e428f606cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:14.448185 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2bdc25-4206-48f2-98b4-c5e428f606cd None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:14.448465 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2bdc25-4206-48f2-98b4-c5e428f606cd None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:14.448732 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2bdc25-4206-48f2-98b4-c5e428f606cd None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:14.449033 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5160d8-7491-402d-abca-36a1205ed584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:14.449368 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2bdc25-4206-48f2-98b4-c5e428f606cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:14.449665 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5160d8-7491-402d-abca-36a1205ed584 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:14.449756 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2bdc25-4206-48f2-98b4-c5e428f606cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:14.449949 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5160d8-7491-402d-abca-36a1205ed584 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:14.450227 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5160d8-7491-402d-abca-36a1205ed584 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:14.451015 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2bdc25-4206-48f2-98b4-c5e428f606cd None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:14.458154 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5160d8-7491-402d-abca-36a1205ed584 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:14.459671 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5160d8-7491-402d-abca-36a1205ed584 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:14.462825 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 376/741] 10.222.0.61 () {56 vars in 1109 bytes} [Mon Aug 22 19:06:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:14.464050 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7b3f33-da44-4289-8a0a-8153be9f7a7e None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:14.464333 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7b3f33-da44-4289-8a0a-8153be9f7a7e None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:14.464600 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7b3f33-da44-4289-8a0a-8153be9f7a7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:14.471254 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-8f5160d8-7491-402d-abca-36a1205ed584 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 22 19:06:14.471254 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:14.471254 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:14.471254 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:14.471254 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:14.471254 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:14.471254 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:14.471254 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:14.471254 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:14.471254 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:14.471254 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:14.471254 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:14.471254 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 19:06:14.471254 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 19:06:14.471254 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 19:06:14.471254 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 19:06:14.472399 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:14.472399 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:14.472399 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 19:06:14.472399 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 19:06:14.472399 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 19:06:14.472399 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 19:06:14.472399 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:14.472399 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:14.472399 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:14.472399 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:14.472399 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:14.472399 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:14.472399 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:14.472399 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:14.472399 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:14.472399 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:14.472399 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:14.472399 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:14.473376 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:14.473376 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:14.473376 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 19:06:14.473376 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 19:06:14.473376 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 19:06:14.473376 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 19:06:14.473376 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 19:06:14.473376 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 19:06:14.473376 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 22 19:06:14.473376 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:06:14.473376 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 366/742] 10.222.0.61 () {56 vars in 1117 bytes} [Mon Aug 22 19:06:14 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 19:06:14.473376 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5160d8-7491-402d-abca-36a1205ed584 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:14.485648 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-ec7b3f33-da44-4289-8a0a-8153be9f7a7e None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:06:14.486827 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-ec7b3f33-da44-4289-8a0a-8153be9f7a7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:06:14.554000 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 377/743] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:06:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 19:06:14.555133 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7b3f33-da44-4289-8a0a-8153be9f7a7e None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:14.573970 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:14.578830 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5160d8-7491-402d-abca-36a1205ed584 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:14.614674 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a95d11-5a94-4b3c-8baf-00689fcbefc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:14.615542 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a95d11-5a94-4b3c-8baf-00689fcbefc5 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:14.616129 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a95d11-5a94-4b3c-8baf-00689fcbefc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:14.616405 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a95d11-5a94-4b3c-8baf-00689fcbefc5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:14.616532 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:14.629054 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a95d11-5a94-4b3c-8baf-00689fcbefc5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 19:06:14.634536 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-12a95d11-5a94-4b3c-8baf-00689fcbefc5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 19:06:14.634536 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:14.634536 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:14.634536 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:14.634536 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:14.634536 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:14.634536 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:14.634536 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:14.634536 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:14.634536 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:14.634536 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:14.634536 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:14.634536 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 19:06:14.634536 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 19:06:14.634536 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 19:06:14.634536 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 19:06:14.635520 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 19:06:14.635520 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 19:06:14.635520 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 19:06:14.635520 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 19:06:14.635520 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:14.635520 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:14.635520 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:14.635520 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:14.635520 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:14.635520 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:14.635520 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:14.635520 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:14.635520 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:14.635520 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:14.635520 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:14.635520 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:14.636745 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:14.636745 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:14.636745 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:14.636745 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:14.636745 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 19:06:14.636745 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 19:06:14.636745 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 19:06:14.636745 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 19:06:14.636745 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 19:06:14.636745 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 19:06:14.636745 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 19:06:14.636745 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:06:14.636745 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 367/744] 10.222.0.61 () {56 vars in 1096 bytes} [Mon Aug 22 19:06:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:14.637471 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a95d11-5a94-4b3c-8baf-00689fcbefc5 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:14.665072 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7b3f33-da44-4289-8a0a-8153be9f7a7e None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:14.703479 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800a9dcd-752a-4003-b887-5e1288d3cede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:14.704120 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800a9dcd-752a-4003-b887-5e1288d3cede None admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:14.704395 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800a9dcd-752a-4003-b887-5e1288d3cede None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:14.704668 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800a9dcd-752a-4003-b887-5e1288d3cede None admin] PATH_INFO: `/v3/services` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:14.705313 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800a9dcd-752a-4003-b887-5e1288d3cede None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:14.706675 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800a9dcd-752a-4003-b887-5e1288d3cede None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:14.724184 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 378/745] 10.222.0.61 () {60 vars in 1141 bytes} [Mon Aug 22 19:06:14 2022] POST /identity/v3/services => generated 240 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:06:14.725059 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800a9dcd-752a-4003-b887-5e1288d3cede None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:14.746974 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a95d11-5a94-4b3c-8baf-00689fcbefc5 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:14.796381 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56325535-e87b-45e4-8a36-7b40bf325cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:14.797659 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56325535-e87b-45e4-8a36-7b40bf325cd4 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:14.798289 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56325535-e87b-45e4-8a36-7b40bf325cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:14.798774 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56325535-e87b-45e4-8a36-7b40bf325cd4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:14.807374 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56325535-e87b-45e4-8a36-7b40bf325cd4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:14.809453 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56325535-e87b-45e4-8a36-7b40bf325cd4 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:14.823234 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-56325535-e87b-45e4-8a36-7b40bf325cd4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 19:06:14.823234 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:14.823234 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:14.823234 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:14.823234 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:14.823234 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:14.823234 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:14.823234 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:14.823234 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:14.823234 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:14.823234 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:14.823234 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:14.823234 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 19:06:14.823234 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 19:06:14.823234 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 19:06:14.824352 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 19:06:14.824352 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:14.824352 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:14.824352 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:14.824352 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:14.824352 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:14.824352 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:14.824352 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:14.824352 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:14.824352 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:14.824352 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:14.824352 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:14.824352 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:14.824352 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:14.824352 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:14.824352 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:14.824352 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:14.825246 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 19:06:14.825246 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 19:06:14.825246 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 19:06:14.825246 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 19:06:14.825246 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 19:06:14.825246 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 19:06:14.825246 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 19:06:14.825246 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:06:14.828196 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 368/746] 10.222.0.61 () {56 vars in 1111 bytes} [Mon Aug 22 19:06:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:14.829458 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56325535-e87b-45e4-8a36-7b40bf325cd4 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:14.849030 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:14.854222 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800a9dcd-752a-4003-b887-5e1288d3cede None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:14.862931 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:14.899822 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc64c2a7-644c-412d-9e0d-07575f7aa91e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:14.900356 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc64c2a7-644c-412d-9e0d-07575f7aa91e None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:14.900632 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc64c2a7-644c-412d-9e0d-07575f7aa91e None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:14.900899 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc64c2a7-644c-412d-9e0d-07575f7aa91e None admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:14.901748 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc64c2a7-644c-412d-9e0d-07575f7aa91e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:14.902186 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc64c2a7-644c-412d-9e0d-07575f7aa91e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:14.903576 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc64c2a7-644c-412d-9e0d-07575f7aa91e None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:14.922350 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 379/747] 10.222.0.61 () {56 vars in 1149 bytes} [Mon Aug 22 19:06:14 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:14.923239 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc64c2a7-644c-412d-9e0d-07575f7aa91e None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:14.952096 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56325535-e87b-45e4-8a36-7b40bf325cd4 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:14.985696 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daadfe78-9976-47a7-82c2-f58fa2385002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:14.986295 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daadfe78-9976-47a7-82c2-f58fa2385002 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:14.986569 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daadfe78-9976-47a7-82c2-f58fa2385002 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:14.986835 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daadfe78-9976-47a7-82c2-f58fa2385002 None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:14.987498 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daadfe78-9976-47a7-82c2-f58fa2385002 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:14.987825 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daadfe78-9976-47a7-82c2-f58fa2385002 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:14.989434 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daadfe78-9976-47a7-82c2-f58fa2385002 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:15.002238 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 369/748] 10.222.0.61 () {56 vars in 1099 bytes} [Mon Aug 22 19:06:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:15.002238 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daadfe78-9976-47a7-82c2-f58fa2385002 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:15.036963 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc64c2a7-644c-412d-9e0d-07575f7aa91e None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:15.074150 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea81c65d-1183-427c-8124-d0d3a79c044f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:15.074840 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea81c65d-1183-427c-8124-d0d3a79c044f None admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:15.075183 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea81c65d-1183-427c-8124-d0d3a79c044f None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:15.075521 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea81c65d-1183-427c-8124-d0d3a79c044f None admin] PATH_INFO: `/v3/services` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:15.076371 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea81c65d-1183-427c-8124-d0d3a79c044f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:15.078115 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:15.078660 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea81c65d-1183-427c-8124-d0d3a79c044f None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:15.095025 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 380/749] 10.222.0.61 () {60 vars in 1141 bytes} [Mon Aug 22 19:06:14 2022] POST /identity/v3/services => generated 251 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:06:15.096042 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea81c65d-1183-427c-8124-d0d3a79c044f None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:15.127469 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daadfe78-9976-47a7-82c2-f58fa2385002 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:15.134349 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:15.191621 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03d5901-b2d8-4e44-b2e2-712d489e1b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:15.192274 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03d5901-b2d8-4e44-b2e2-712d489e1b8a None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:15.192640 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03d5901-b2d8-4e44-b2e2-712d489e1b8a None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:15.193200 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03d5901-b2d8-4e44-b2e2-712d489e1b8a None admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:15.194286 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03d5901-b2d8-4e44-b2e2-712d489e1b8a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:15.195324 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03d5901-b2d8-4e44-b2e2-712d489e1b8a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:15.197228 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03d5901-b2d8-4e44-b2e2-712d489e1b8a None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:15.219846 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 370/750] 10.222.0.61 () {56 vars in 1148 bytes} [Mon Aug 22 19:06:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:15.220752 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03d5901-b2d8-4e44-b2e2-712d489e1b8a None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:15.236456 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea81c65d-1183-427c-8124-d0d3a79c044f None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:15.274788 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e77ad5b-04db-4bdb-9dd4-654e6abac616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:15.275552 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e77ad5b-04db-4bdb-9dd4-654e6abac616 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:15.275956 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e77ad5b-04db-4bdb-9dd4-654e6abac616 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:15.276304 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e77ad5b-04db-4bdb-9dd4-654e6abac616 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:15.342316 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03d5901-b2d8-4e44-b2e2-712d489e1b8a None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:15.367627 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e77ad5b-04db-4bdb-9dd4-654e6abac616 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:15.369703 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e77ad5b-04db-4bdb-9dd4-654e6abac616 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:15.381852 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:15.386771 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b4028c-781f-4504-a208-9ae02809663e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:15.387587 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b4028c-781f-4504-a208-9ae02809663e None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:15.388369 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b4028c-781f-4504-a208-9ae02809663e None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:15.388809 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b4028c-781f-4504-a208-9ae02809663e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:15.401244 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:15.404118 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b4028c-781f-4504-a208-9ae02809663e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:15.408215 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b4028c-781f-4504-a208-9ae02809663e None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:15.427518 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-e9b4028c-781f-4504-a208-9ae02809663e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:06:15.427518 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:15.427518 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:15.427518 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:15.427518 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:15.427518 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:15.427518 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:15.427518 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:15.427518 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:15.427518 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:15.427518 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:15.427518 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:15.427518 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 19:06:15.427518 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 19:06:15.427518 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 19:06:15.427518 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 19:06:15.428693 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:15.428693 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:15.428693 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:15.428693 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:15.428693 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:15.428693 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:15.428693 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:15.428693 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:15.428693 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:15.428693 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:15.428693 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:15.428693 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:15.428693 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:15.428693 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:15.428693 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:15.428693 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:15.428693 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 19:06:15.429790 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 19:06:15.429790 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:06:15.429790 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:06:15.432228 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 371/751] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:06:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:15.433403 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b4028c-781f-4504-a208-9ae02809663e None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:15.499083 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 381/752] 10.222.0.61 () {58 vars in 1282 bytes} [Mon Aug 22 19:06:15 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 22 19:06:15.500293 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e77ad5b-04db-4bdb-9dd4-654e6abac616 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:15.555116 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b4028c-781f-4504-a208-9ae02809663e None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:15.590330 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86204e5b-bf0d-4270-9b6a-d2099288480d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:15.590986 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86204e5b-bf0d-4270-9b6a-d2099288480d None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:15.591205 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86204e5b-bf0d-4270-9b6a-d2099288480d None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:15.591473 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86204e5b-bf0d-4270-9b6a-d2099288480d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:15.592393 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86204e5b-bf0d-4270-9b6a-d2099288480d None admin] RBAC: Adding query filter params (role.id=163c30c3b3c246beb849ce176f02d9da, scope.project.id=e07dc5ebdc504c6da706a2d21c0a3970, user.id=2824359491934d0dacaa21a718becee5) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:15.592529 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86204e5b-bf0d-4270-9b6a-d2099288480d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:15.594514 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86204e5b-bf0d-4270-9b6a-d2099288480d None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:15.600661 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 372/753] 10.222.0.61 () {56 vars in 1374 bytes} [Mon Aug 22 19:06:15 2022] GET /identity/v3/role_assignments?user.id=2824359491934d0dacaa21a718becee5&scope.project.id=e07dc5ebdc504c6da706a2d21c0a3970&role.id=163c30c3b3c246beb849ce176f02d9da => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:15.601554 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86204e5b-bf0d-4270-9b6a-d2099288480d None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:15.611277 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e77ad5b-04db-4bdb-9dd4-654e6abac616 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:15.661714 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:15.679666 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:15.689566 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8204a43-ca72-4c4d-a6f9-647bd5967050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:15.690525 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8204a43-ca72-4c4d-a6f9-647bd5967050 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:15.691217 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8204a43-ca72-4c4d-a6f9-647bd5967050 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:15.691634 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8204a43-ca72-4c4d-a6f9-647bd5967050 None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:15.692461 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8204a43-ca72-4c4d-a6f9-647bd5967050 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:15.692907 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8204a43-ca72-4c4d-a6f9-647bd5967050 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:15.694454 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8204a43-ca72-4c4d-a6f9-647bd5967050 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:15.719222 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86204e5b-bf0d-4270-9b6a-d2099288480d None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:15.723140 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 382/754] 10.222.0.61 () {56 vars in 1109 bytes} [Mon Aug 22 19:06:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:15.737130 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8204a43-ca72-4c4d-a6f9-647bd5967050 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:15.761384 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b01262b-6dbb-48fe-ae93-6648b8ba11b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:15.761942 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b01262b-6dbb-48fe-ae93-6648b8ba11b4 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:15.762216 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b01262b-6dbb-48fe-ae93-6648b8ba11b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:15.762484 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b01262b-6dbb-48fe-ae93-6648b8ba11b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:15.775439 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b01262b-6dbb-48fe-ae93-6648b8ba11b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:15.776966 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b01262b-6dbb-48fe-ae93-6648b8ba11b4 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:15.792890 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-7b01262b-6dbb-48fe-ae93-6648b8ba11b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:06:15.792890 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:15.792890 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:15.792890 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:15.792890 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:15.792890 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:15.792890 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:15.792890 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:15.792890 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:15.792890 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:15.792890 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:15.792890 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:15.792890 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 19:06:15.792890 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 19:06:15.792890 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 19:06:15.792890 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 19:06:15.793906 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:15.793906 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:15.793906 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:15.793906 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:15.793906 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:15.793906 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:15.793906 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:15.793906 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:15.793906 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:15.793906 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:15.793906 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:15.793906 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:15.793906 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:15.793906 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:15.793906 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:15.793906 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:15.793906 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 19:06:15.794825 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 19:06:15.794825 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:06:15.794825 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:06:15.798383 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 373/755] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:06:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:15.799663 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b01262b-6dbb-48fe-ae93-6648b8ba11b4 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:15.877807 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8204a43-ca72-4c4d-a6f9-647bd5967050 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:15.904058 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:15.907099 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b01262b-6dbb-48fe-ae93-6648b8ba11b4 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:15.913790 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15394b86-e65d-4bf4-9bd2-b2b7b41e8893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:15.914431 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15394b86-e65d-4bf4-9bd2-b2b7b41e8893 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:15.914713 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15394b86-e65d-4bf4-9bd2-b2b7b41e8893 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:15.914986 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15394b86-e65d-4bf4-9bd2-b2b7b41e8893 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:15.923070 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15394b86-e65d-4bf4-9bd2-b2b7b41e8893 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:15.924538 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15394b86-e65d-4bf4-9bd2-b2b7b41e8893 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:15.936154 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-15394b86-e65d-4bf4-9bd2-b2b7b41e8893 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 22 19:06:15.936154 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:15.936154 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:15.936154 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:15.936154 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:15.936154 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:15.936154 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:15.936154 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:15.936154 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:15.936154 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:15.936154 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:15.936154 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:15.936154 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 19:06:15.936154 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 19:06:15.936154 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 19:06:15.936154 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 19:06:15.937235 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:15.937235 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:15.937235 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 19:06:15.937235 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 19:06:15.937235 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 19:06:15.937235 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 19:06:15.937235 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:15.937235 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:15.937235 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:15.937235 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:15.937235 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:15.937235 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:15.937235 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:15.937235 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:15.937235 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:15.937235 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:15.937235 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:15.937235 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:15.938225 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:15.938225 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:15.938225 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 19:06:15.938225 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 19:06:15.938225 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 19:06:15.938225 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 19:06:15.938225 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 19:06:15.938225 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 19:06:15.938225 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 22 19:06:15.938225 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:06:15.938225 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 383/756] 10.222.0.61 () {56 vars in 1093 bytes} [Mon Aug 22 19:06:15 2022] GET /identity/v3/users/nova => generated 82 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 19:06:15.942572 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15394b86-e65d-4bf4-9bd2-b2b7b41e8893 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:15.950688 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579b02cf-e35e-498e-99d5-13d35bca26d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:15.951305 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579b02cf-e35e-498e-99d5-13d35bca26d4 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:15.951579 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579b02cf-e35e-498e-99d5-13d35bca26d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:15.951878 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579b02cf-e35e-498e-99d5-13d35bca26d4 None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:15.953067 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579b02cf-e35e-498e-99d5-13d35bca26d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:15.953363 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579b02cf-e35e-498e-99d5-13d35bca26d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:15.954740 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579b02cf-e35e-498e-99d5-13d35bca26d4 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:15.989887 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 374/757] 10.222.0.61 () {56 vars in 1109 bytes} [Mon Aug 22 19:06:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:16.005148 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579b02cf-e35e-498e-99d5-13d35bca26d4 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:16.039169 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:16.122382 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15394b86-e65d-4bf4-9bd2-b2b7b41e8893 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:16.161065 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce808ba5-5e85-42f8-ba03-edefc888bf5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:16.161689 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce808ba5-5e85-42f8-ba03-edefc888bf5b None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:16.161973 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce808ba5-5e85-42f8-ba03-edefc888bf5b None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:16.162250 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce808ba5-5e85-42f8-ba03-edefc888bf5b None admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:16.163009 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce808ba5-5e85-42f8-ba03-edefc888bf5b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:16.163322 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce808ba5-5e85-42f8-ba03-edefc888bf5b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:16.164945 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce808ba5-5e85-42f8-ba03-edefc888bf5b None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:16.185593 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 384/758] 10.222.0.61 () {56 vars in 1133 bytes} [Mon Aug 22 19:06:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 19:06:16.191886 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce808ba5-5e85-42f8-ba03-edefc888bf5b None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:16.252512 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579b02cf-e35e-498e-99d5-13d35bca26d4 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:16.322495 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001fc23d-1ab7-4700-b8ac-13d75b2ea53b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:16.323120 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001fc23d-1ab7-4700-b8ac-13d75b2ea53b None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:16.323396 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001fc23d-1ab7-4700-b8ac-13d75b2ea53b None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:16.323684 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001fc23d-1ab7-4700-b8ac-13d75b2ea53b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:16.337079 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001fc23d-1ab7-4700-b8ac-13d75b2ea53b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:16.338530 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001fc23d-1ab7-4700-b8ac-13d75b2ea53b None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:16.345385 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce808ba5-5e85-42f8-ba03-edefc888bf5b None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:16.355987 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:16.358152 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-001fc23d-1ab7-4700-b8ac-13d75b2ea53b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 19:06:16.358152 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:16.358152 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:16.358152 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:16.358152 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:16.358152 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:16.358152 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:16.358152 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:16.358152 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:16.358152 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:16.358152 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:16.358152 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:16.358152 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 19:06:16.358152 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 19:06:16.358152 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 19:06:16.359744 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 19:06:16.359744 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:16.359744 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:16.359744 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:16.359744 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:16.359744 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:16.359744 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:16.359744 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:16.359744 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:16.359744 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:16.359744 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:16.359744 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:16.359744 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:16.359744 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:16.359744 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:16.359744 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:16.359744 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:16.360669 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 19:06:16.360669 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 19:06:16.360669 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 19:06:16.360669 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 19:06:16.360669 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 19:06:16.360669 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 19:06:16.360669 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 19:06:16.360669 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:06:16.363116 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 375/759] 10.222.0.61 () {56 vars in 1111 bytes} [Mon Aug 22 19:06:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 357 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 19:06:16.367397 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001fc23d-1ab7-4700-b8ac-13d75b2ea53b None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:16.393163 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:16.425263 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba37160-0bd2-4a11-adee-63c624cda81a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:16.425933 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba37160-0bd2-4a11-adee-63c624cda81a None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:16.426214 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba37160-0bd2-4a11-adee-63c624cda81a None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:16.426490 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba37160-0bd2-4a11-adee-63c624cda81a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:16.456671 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba37160-0bd2-4a11-adee-63c624cda81a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:16.458134 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba37160-0bd2-4a11-adee-63c624cda81a None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:16.481033 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001fc23d-1ab7-4700-b8ac-13d75b2ea53b None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:16.490411 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-5ba37160-0bd2-4a11-adee-63c624cda81a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:06:16.490411 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:16.490411 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:16.490411 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:16.490411 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:16.490411 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:16.490411 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:16.490411 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:16.490411 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:16.490411 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:16.490411 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:16.490411 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:16.490411 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 19:06:16.490411 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 19:06:16.490411 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 19:06:16.490411 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 19:06:16.491328 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:16.491328 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:16.491328 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:16.491328 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:16.491328 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:16.491328 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:16.491328 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:16.491328 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:16.491328 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:16.491328 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:16.491328 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:16.491328 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:16.491328 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:16.491328 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:16.491328 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:16.491328 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:16.491328 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 19:06:16.492257 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 19:06:16.492257 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:06:16.492257 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:06:16.495731 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 385/760] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:06:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:16.505050 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba37160-0bd2-4a11-adee-63c624cda81a None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:16.555116 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35915c6-43b3-46bc-8103-23e98eff1a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:16.555819 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35915c6-43b3-46bc-8103-23e98eff1a8e None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:16.556102 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35915c6-43b3-46bc-8103-23e98eff1a8e None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:16.557604 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35915c6-43b3-46bc-8103-23e98eff1a8e None admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:16.558321 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35915c6-43b3-46bc-8103-23e98eff1a8e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:16.558634 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35915c6-43b3-46bc-8103-23e98eff1a8e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:16.560098 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35915c6-43b3-46bc-8103-23e98eff1a8e None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:16.587502 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 376/761] 10.222.0.61 () {56 vars in 1148 bytes} [Mon Aug 22 19:06:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 19:06:16.597804 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35915c6-43b3-46bc-8103-23e98eff1a8e None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:16.680407 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba37160-0bd2-4a11-adee-63c624cda81a None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:16.742526 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:16.748672 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa282963-2186-4dd6-9782-e6265809287f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:16.750553 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa282963-2186-4dd6-9782-e6265809287f None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:16.751031 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa282963-2186-4dd6-9782-e6265809287f None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:16.751895 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa282963-2186-4dd6-9782-e6265809287f None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:16.753964 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa282963-2186-4dd6-9782-e6265809287f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:16.754691 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa282963-2186-4dd6-9782-e6265809287f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:16.756859 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa282963-2186-4dd6-9782-e6265809287f None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:16.778926 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 386/762] 10.222.0.61 () {56 vars in 1109 bytes} [Mon Aug 22 19:06:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 19:06:16.782599 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa282963-2186-4dd6-9782-e6265809287f None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:16.787496 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35915c6-43b3-46bc-8103-23e98eff1a8e None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:16.811106 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:16.869209 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05550063-c78e-4386-879e-8712ac2ad1fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:16.870215 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05550063-c78e-4386-879e-8712ac2ad1fd None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:16.870739 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05550063-c78e-4386-879e-8712ac2ad1fd None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:16.871324 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05550063-c78e-4386-879e-8712ac2ad1fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:16.872096 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05550063-c78e-4386-879e-8712ac2ad1fd None admin] RBAC: Adding query filter params (role.id=324bd04d55c34b61a5282c9f6ed73f46, scope.project.id=e07dc5ebdc504c6da706a2d21c0a3970, user.id=399396a8fe254da199bc48b0ca5243d0) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:16.872429 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05550063-c78e-4386-879e-8712ac2ad1fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:16.873959 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05550063-c78e-4386-879e-8712ac2ad1fd None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:16.889250 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 377/763] 10.222.0.61 () {56 vars in 1374 bytes} [Mon Aug 22 19:06:16 2022] GET /identity/v3/role_assignments?user.id=399396a8fe254da199bc48b0ca5243d0&scope.project.id=e07dc5ebdc504c6da706a2d21c0a3970&role.id=324bd04d55c34b61a5282c9f6ed73f46 => generated 259 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 19:06:16.925883 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa282963-2186-4dd6-9782-e6265809287f None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:16.963459 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67922a0b-5a74-431e-84c7-a91b5202c8c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:16.964468 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67922a0b-5a74-431e-84c7-a91b5202c8c1 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:16.965011 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67922a0b-5a74-431e-84c7-a91b5202c8c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:16.965484 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67922a0b-5a74-431e-84c7-a91b5202c8c1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:16.972558 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67922a0b-5a74-431e-84c7-a91b5202c8c1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:16.974388 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67922a0b-5a74-431e-84c7-a91b5202c8c1 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:16.987332 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-67922a0b-5a74-431e-84c7-a91b5202c8c1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 19:06:16.987332 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:16.987332 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:16.987332 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:16.987332 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:16.987332 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:16.987332 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:16.987332 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:16.987332 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:16.987332 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:16.987332 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:16.987332 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:16.987332 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 19:06:16.987332 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 19:06:16.987332 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 19:06:16.988365 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 19:06:16.988365 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:16.988365 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:16.988365 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:16.988365 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:16.988365 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:16.988365 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:16.988365 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:16.988365 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:16.988365 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:16.988365 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:16.988365 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:16.988365 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:16.988365 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:16.988365 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:16.988365 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:16.988365 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:16.989571 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 19:06:16.989571 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 19:06:16.989571 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 19:06:16.989571 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 19:06:16.989571 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 19:06:16.989571 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 19:06:16.989571 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 19:06:16.989571 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:06:16.992401 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 387/764] 10.222.0.61 () {56 vars in 1111 bytes} [Mon Aug 22 19:06:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:16.997857 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05550063-c78e-4386-879e-8712ac2ad1fd None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:17.010176 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed54cf0-d510-4580-be12-1af9cd0b82ad None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:17.010511 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed54cf0-d510-4580-be12-1af9cd0b82ad None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:17.010721 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed54cf0-d510-4580-be12-1af9cd0b82ad None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:17.012493 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 388/765] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:06:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 19:06:17.018760 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a71b935-ef73-4a13-9920-3a365a7420aa None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:17.023738 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a71b935-ef73-4a13-9920-3a365a7420aa None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:17.023738 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a71b935-ef73-4a13-9920-3a365a7420aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:17.053438 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-6a71b935-ef73-4a13-9920-3a365a7420aa None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:06:17.057173 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-6a71b935-ef73-4a13-9920-3a365a7420aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:06:17.165580 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 389/766] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:06:17 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 22 19:06:17.179056 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05550063-c78e-4386-879e-8712ac2ad1fd None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:17.229376 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773a7c6c-cab1-482a-b1b4-a7bc4b45a335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:17.230002 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773a7c6c-cab1-482a-b1b4-a7bc4b45a335 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:17.230288 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773a7c6c-cab1-482a-b1b4-a7bc4b45a335 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:17.230565 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773a7c6c-cab1-482a-b1b4-a7bc4b45a335 None admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:17.231274 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773a7c6c-cab1-482a-b1b4-a7bc4b45a335 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:17.231583 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773a7c6c-cab1-482a-b1b4-a7bc4b45a335 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:17.233037 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773a7c6c-cab1-482a-b1b4-a7bc4b45a335 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:17.250754 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 378/767] 10.222.0.61 () {56 vars in 1148 bytes} [Mon Aug 22 19:06:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:17.256258 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a71b935-ef73-4a13-9920-3a365a7420aa None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:17.259943 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:17.327117 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0a7626-ad58-49dc-8249-0603a4bd5303 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:17.327437 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0a7626-ad58-49dc-8249-0603a4bd5303 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:17.327747 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0a7626-ad58-49dc-8249-0603a4bd5303 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:17.329205 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 379/768] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:06:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:06:17.334203 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475e3edb-c881-434d-9660-fc7714fd11c2 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:17.334486 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475e3edb-c881-434d-9660-fc7714fd11c2 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:17.334759 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475e3edb-c881-434d-9660-fc7714fd11c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:17.351410 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:17.376157 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-475e3edb-c881-434d-9660-fc7714fd11c2 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:06:17.377359 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-475e3edb-c881-434d-9660-fc7714fd11c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:06:17.384721 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a71b935-ef73-4a13-9920-3a365a7420aa None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:17.421812 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203e7dc7-c5dc-4093-954a-c42dbfaaeef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:17.422405 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203e7dc7-c5dc-4093-954a-c42dbfaaeef4 None admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:17.422683 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203e7dc7-c5dc-4093-954a-c42dbfaaeef4 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:17.422963 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203e7dc7-c5dc-4093-954a-c42dbfaaeef4 None admin] PATH_INFO: `/v3/projects/e07dc5ebdc504c6da706a2d21c0a3970/users/e7f43e2756b746a6825ca1a06b7d7c91/roles/163c30c3b3c246beb849ce176f02d9da` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:17.425055 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203e7dc7-c5dc-4093-954a-c42dbfaaeef4 None admin] RBAC: Authorizing `identity:create_grant(project_id=e07dc5ebdc504c6da706a2d21c0a3970, user_id=e7f43e2756b746a6825ca1a06b7d7c91, role_id=163c30c3b3c246beb849ce176f02d9da)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:17.426488 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203e7dc7-c5dc-4093-954a-c42dbfaaeef4 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:17.444162 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 390/769] 10.222.0.61 () {58 vars in 1439 bytes} [Mon Aug 22 19:06:17 2022] PUT /identity/v3/projects/e07dc5ebdc504c6da706a2d21c0a3970/users/e7f43e2756b746a6825ca1a06b7d7c91/roles/163c30c3b3c246beb849ce176f02d9da => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:06:17.518429 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203e7dc7-c5dc-4093-954a-c42dbfaaeef4 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:17.532603 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 380/770] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:06:17 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 19:06:17.592030 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:17.634464 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203e7dc7-c5dc-4093-954a-c42dbfaaeef4 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:17.674541 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdafcc4-4401-4bc5-b2a7-1adcd3d46b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:17.675110 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdafcc4-4401-4bc5-b2a7-1adcd3d46b29 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:17.675390 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdafcc4-4401-4bc5-b2a7-1adcd3d46b29 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:17.675684 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdafcc4-4401-4bc5-b2a7-1adcd3d46b29 None admin] PATH_INFO: `/v3/services/network` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:17.676295 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdafcc4-4401-4bc5-b2a7-1adcd3d46b29 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:17.677687 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdafcc4-4401-4bc5-b2a7-1adcd3d46b29 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:17.688103 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-dcdafcc4-4401-4bc5-b2a7-1adcd3d46b29 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 22 19:06:17.688103 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:17.688103 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 19:06:17.688103 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 19:06:17.688103 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 19:06:17.688103 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 19:06:17.688103 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 19:06:17.688103 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 19:06:17.688103 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 22 19:06:17.688103 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:06:17.688103 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 19:06:17.688103 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:06:17.688103 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:17.688103 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:17.688103 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:17.688103 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:17.688103 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:17.689126 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:17.689126 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:17.689126 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:17.689126 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:17.689126 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:17.689126 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:17.689126 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 19:06:17.689126 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 19:06:17.689126 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 19:06:17.689126 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 19:06:17.689126 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:17.689126 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:17.689126 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:17.689126 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:17.689126 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:17.689126 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:17.689967 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:17.689967 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:17.689967 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:17.689967 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:17.689967 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:17.689967 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:17.689967 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:17.689967 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:17.689967 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:17.689967 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:17.689967 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 19:06:17.689967 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 19:06:17.689967 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 22 19:06:17.689967 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:06:17.689967 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 391/771] 10.222.0.61 () {56 vars in 1111 bytes} [Mon Aug 22 19:06:17 2022] GET /identity/v3/services/network => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 19:06:17.695445 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475e3edb-c881-434d-9660-fc7714fd11c2 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:17.779127 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5172be4e-6f66-4c49-9dda-28af359b769b None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:17.779448 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5172be4e-6f66-4c49-9dda-28af359b769b None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:17.780028 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5172be4e-6f66-4c49-9dda-28af359b769b None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:17.781882 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 392/772] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:06:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 19:06:17.786681 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282c0153-0b16-4409-abfe-069f56e41f68 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:17.786971 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282c0153-0b16-4409-abfe-069f56e41f68 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:17.787244 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282c0153-0b16-4409-abfe-069f56e41f68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:17.806172 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475e3edb-c881-434d-9660-fc7714fd11c2 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:17.820670 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-282c0153-0b16-4409-abfe-069f56e41f68 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:06:17.821942 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-282c0153-0b16-4409-abfe-069f56e41f68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:06:17.842589 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29370a47-e8ed-4e71-b2fa-b50bbb4f85a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:17.844076 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29370a47-e8ed-4e71-b2fa-b50bbb4f85a4 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:17.844579 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29370a47-e8ed-4e71-b2fa-b50bbb4f85a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:17.845068 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29370a47-e8ed-4e71-b2fa-b50bbb4f85a4 None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:17.846026 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29370a47-e8ed-4e71-b2fa-b50bbb4f85a4 None admin] RBAC: Adding query filter params (name=network) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:17.846577 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29370a47-e8ed-4e71-b2fa-b50bbb4f85a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:17.848171 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29370a47-e8ed-4e71-b2fa-b50bbb4f85a4 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:17.853559 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 381/773] 10.222.0.61 () {56 vars in 1112 bytes} [Mon Aug 22 19:06:17 2022] GET /identity/v3/services?name=network => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:17.855452 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:17.859260 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29370a47-e8ed-4e71-b2fa-b50bbb4f85a4 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:17.929226 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 393/774] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:06:17 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 19:06:17.930250 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282c0153-0b16-4409-abfe-069f56e41f68 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:17.996836 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:18.037614 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29370a47-e8ed-4e71-b2fa-b50bbb4f85a4 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:18.045092 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282c0153-0b16-4409-abfe-069f56e41f68 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:18.083660 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62584ef-d9ba-4021-af14-5a55f0728146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:18.084445 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62584ef-d9ba-4021-af14-5a55f0728146 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:18.084720 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62584ef-d9ba-4021-af14-5a55f0728146 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:18.084988 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62584ef-d9ba-4021-af14-5a55f0728146 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:18.085729 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62584ef-d9ba-4021-af14-5a55f0728146 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:18.087148 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62584ef-d9ba-4021-af14-5a55f0728146 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:18.096982 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-d62584ef-d9ba-4021-af14-5a55f0728146 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 22 19:06:18.096982 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:18.096982 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 19:06:18.096982 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 19:06:18.096982 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 19:06:18.096982 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 19:06:18.096982 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 19:06:18.096982 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 19:06:18.096982 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 22 19:06:18.096982 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:06:18.096982 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 19:06:18.096982 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:06:18.096982 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:18.096982 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:18.096982 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:18.096982 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:18.096982 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:18.098009 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:18.098009 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:18.098009 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:18.098009 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:18.098009 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:18.098009 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:18.098009 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 19:06:18.098009 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 19:06:18.098009 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 19:06:18.098009 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 19:06:18.098009 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:18.098009 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:18.098009 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:18.098009 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:18.098009 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:18.098009 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:18.098857 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:18.098857 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:18.098857 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:18.098857 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:18.098857 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:18.098857 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:18.098857 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:18.098857 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:18.098857 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:18.098857 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:18.098857 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 19:06:18.098857 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 19:06:18.098857 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 22 19:06:18.098857 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:06:18.102233 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 394/775] 10.222.0.61 () {56 vars in 1129 bytes} [Mon Aug 22 19:06:17 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:18.104381 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62584ef-d9ba-4021-af14-5a55f0728146 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:18.164970 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e6a7ec-aaeb-443a-8412-8b24797084b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:18.165936 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e6a7ec-aaeb-443a-8412-8b24797084b9 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:18.166450 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e6a7ec-aaeb-443a-8412-8b24797084b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:18.166923 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e6a7ec-aaeb-443a-8412-8b24797084b9 None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:18.167797 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e6a7ec-aaeb-443a-8412-8b24797084b9 None admin] RBAC: Adding query filter params (type=network) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:18.168337 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e6a7ec-aaeb-443a-8412-8b24797084b9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:18.169907 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e6a7ec-aaeb-443a-8412-8b24797084b9 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:18.189882 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 382/776] 10.222.0.61 () {56 vars in 1112 bytes} [Mon Aug 22 19:06:17 2022] GET /identity/v3/services?type=network => generated 350 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 19:06:18.196617 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e6a7ec-aaeb-443a-8412-8b24797084b9 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:18.219946 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62584ef-d9ba-4021-af14-5a55f0728146 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:18.245801 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:18.261877 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df6f1d4-0f33-44df-9d74-0a28e4c0d8c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:18.262499 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df6f1d4-0f33-44df-9d74-0a28e4c0d8c7 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:18.262814 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df6f1d4-0f33-44df-9d74-0a28e4c0d8c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:18.263088 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df6f1d4-0f33-44df-9d74-0a28e4c0d8c7 None admin] PATH_INFO: `/v3/services` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:18.267667 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df6f1d4-0f33-44df-9d74-0a28e4c0d8c7 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:18.267667 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df6f1d4-0f33-44df-9d74-0a28e4c0d8c7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:18.267667 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df6f1d4-0f33-44df-9d74-0a28e4c0d8c7 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:18.278841 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 395/777] 10.222.0.61 () {56 vars in 1124 bytes} [Mon Aug 22 19:06:18 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 19:06:18.279785 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df6f1d4-0f33-44df-9d74-0a28e4c0d8c7 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:18.308035 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:18.342765 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e6a7ec-aaeb-443a-8412-8b24797084b9 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:18.378990 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ed9b23-9c36-4657-ad1f-4886e6dc8ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:18.379680 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ed9b23-9c36-4657-ad1f-4886e6dc8ba3 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:18.380060 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ed9b23-9c36-4657-ad1f-4886e6dc8ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:18.380438 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ed9b23-9c36-4657-ad1f-4886e6dc8ba3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:18.381151 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ed9b23-9c36-4657-ad1f-4886e6dc8ba3 None admin] RBAC: Adding query filter params (interface=public, service_id=a7daf8f2682645608e1f6c80ca76a96e, region_id=RegionOne) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:18.381462 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ed9b23-9c36-4657-ad1f-4886e6dc8ba3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:18.382922 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ed9b23-9c36-4657-ad1f-4886e6dc8ba3 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:18.392205 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 383/778] 10.222.0.61 () {56 vars in 1251 bytes} [Mon Aug 22 19:06:18 2022] GET /identity/v3/endpoints?service_id=a7daf8f2682645608e1f6c80ca76a96e&interface=public®ion_id=RegionOne => generated 194 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:18.393078 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ed9b23-9c36-4657-ad1f-4886e6dc8ba3 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:18.405174 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df6f1d4-0f33-44df-9d74-0a28e4c0d8c7 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:18.448391 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43959620-3645-4df5-bd08-059c0972d2d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:18.449105 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43959620-3645-4df5-bd08-059c0972d2d4 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:18.449328 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43959620-3645-4df5-bd08-059c0972d2d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:18.449591 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43959620-3645-4df5-bd08-059c0972d2d4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:18.460905 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43959620-3645-4df5-bd08-059c0972d2d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 19:06:18.466662 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-43959620-3645-4df5-bd08-059c0972d2d4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 19:06:18.466662 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:18.466662 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:18.466662 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:18.466662 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:18.466662 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:18.466662 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:18.466662 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:18.466662 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:18.466662 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:18.466662 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:18.466662 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:18.466662 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 19:06:18.466662 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 19:06:18.466662 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 19:06:18.466662 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 19:06:18.467578 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 19:06:18.467578 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 19:06:18.467578 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 19:06:18.467578 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 19:06:18.467578 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:18.467578 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:18.467578 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:18.467578 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:18.467578 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:18.467578 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:18.467578 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:18.467578 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:18.467578 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:18.467578 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:18.467578 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:18.467578 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:18.469047 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:18.469047 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:18.469047 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:18.469047 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:18.469047 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 19:06:18.469047 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 19:06:18.469047 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 19:06:18.469047 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 19:06:18.469047 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 19:06:18.469047 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 19:06:18.469047 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 19:06:18.469047 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:06:18.473250 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 396/779] 10.222.0.61 () {56 vars in 1096 bytes} [Mon Aug 22 19:06:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:18.474224 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43959620-3645-4df5-bd08-059c0972d2d4 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:18.514293 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ed9b23-9c36-4657-ad1f-4886e6dc8ba3 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:18.527134 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:18.543235 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:18.553599 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105e9892-1ed6-4d24-998a-e945ba25b22d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:18.554515 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105e9892-1ed6-4d24-998a-e945ba25b22d None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:18.554936 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105e9892-1ed6-4d24-998a-e945ba25b22d None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:18.555348 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105e9892-1ed6-4d24-998a-e945ba25b22d None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:18.556163 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105e9892-1ed6-4d24-998a-e945ba25b22d None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:18.556621 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105e9892-1ed6-4d24-998a-e945ba25b22d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:18.558118 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105e9892-1ed6-4d24-998a-e945ba25b22d None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:18.568508 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 384/780] 10.222.0.61 () {56 vars in 1124 bytes} [Mon Aug 22 19:06:18 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:18.569531 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105e9892-1ed6-4d24-998a-e945ba25b22d None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:18.586701 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43959620-3645-4df5-bd08-059c0972d2d4 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:18.623293 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eab12b4-8478-4832-a267-8568b5d1996d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:18.624033 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eab12b4-8478-4832-a267-8568b5d1996d None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:18.624400 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eab12b4-8478-4832-a267-8568b5d1996d None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:18.624685 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eab12b4-8478-4832-a267-8568b5d1996d None admin] PATH_INFO: `/v3/services` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:18.625400 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eab12b4-8478-4832-a267-8568b5d1996d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:18.626784 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eab12b4-8478-4832-a267-8568b5d1996d None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:18.632404 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 397/781] 10.222.0.61 () {56 vars in 1087 bytes} [Mon Aug 22 19:06:18 2022] GET /identity/v3/services => generated 772 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:18.633220 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eab12b4-8478-4832-a267-8568b5d1996d None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:18.685779 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105e9892-1ed6-4d24-998a-e945ba25b22d None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:18.725356 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfbe634-2f03-435f-a809-c82e9aa8e44e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:18.726770 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfbe634-2f03-435f-a809-c82e9aa8e44e None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:18.727224 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfbe634-2f03-435f-a809-c82e9aa8e44e None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:18.727750 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfbe634-2f03-435f-a809-c82e9aa8e44e None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:18.728642 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfbe634-2f03-435f-a809-c82e9aa8e44e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:18.729173 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfbe634-2f03-435f-a809-c82e9aa8e44e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:18.730921 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfbe634-2f03-435f-a809-c82e9aa8e44e None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:18.741676 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 385/782] 10.222.0.61 () {56 vars in 1099 bytes} [Mon Aug 22 19:06:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:18.753966 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfbe634-2f03-435f-a809-c82e9aa8e44e None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:18.757309 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eab12b4-8478-4832-a267-8568b5d1996d None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:18.772035 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:18.801560 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d5b504-9226-4b6e-b7ff-1537d27a61f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:18.802097 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d5b504-9226-4b6e-b7ff-1537d27a61f7 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:18.802380 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d5b504-9226-4b6e-b7ff-1537d27a61f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:18.802652 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d5b504-9226-4b6e-b7ff-1537d27a61f7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:18.803378 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d5b504-9226-4b6e-b7ff-1537d27a61f7 None admin] RBAC: Adding query filter params (interface=public, service_id=e6261a5e208a4592a64da7efdbdd86b9, region_id=RegionOne) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:18.803993 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d5b504-9226-4b6e-b7ff-1537d27a61f7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:18.805349 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d5b504-9226-4b6e-b7ff-1537d27a61f7 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:18.825399 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 398/783] 10.222.0.61 () {56 vars in 1251 bytes} [Mon Aug 22 19:06:18 2022] GET /identity/v3/endpoints?service_id=e6261a5e208a4592a64da7efdbdd86b9&interface=public®ion_id=RegionOne => generated 194 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:18.830655 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d5b504-9226-4b6e-b7ff-1537d27a61f7 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:18.867273 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:18.924139 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfbe634-2f03-435f-a809-c82e9aa8e44e None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:18.962815 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d5b504-9226-4b6e-b7ff-1537d27a61f7 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:18.963507 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc08779-6108-40aa-b2fd-3e5ea10594b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:18.964276 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc08779-6108-40aa-b2fd-3e5ea10594b1 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:18.964620 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc08779-6108-40aa-b2fd-3e5ea10594b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:18.964957 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc08779-6108-40aa-b2fd-3e5ea10594b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:18.979805 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc08779-6108-40aa-b2fd-3e5ea10594b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:18.981349 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc08779-6108-40aa-b2fd-3e5ea10594b1 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:19.003046 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-afc08779-6108-40aa-b2fd-3e5ea10594b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:06:19.003046 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:19.003046 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:19.003046 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:19.003046 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:19.003046 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:19.003046 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:19.003046 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:19.003046 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:19.003046 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:19.003046 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:19.003046 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:19.003046 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 19:06:19.003046 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 19:06:19.003046 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 19:06:19.003046 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 19:06:19.004119 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:19.004119 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:19.004119 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:19.004119 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:19.004119 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:19.004119 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:19.004119 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:19.004119 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:19.004119 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:19.004119 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:19.004119 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:19.004119 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:19.004119 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:19.004119 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:19.004119 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:19.004119 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:19.004119 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 19:06:19.005024 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 19:06:19.005024 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:06:19.005024 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:06:19.005638 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 386/784] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:06:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 19:06:19.008389 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6de3480-97e0-4286-8f5b-ad67a7268bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:19.009140 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6de3480-97e0-4286-8f5b-ad67a7268bdf None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:19.009418 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6de3480-97e0-4286-8f5b-ad67a7268bdf None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:19.009692 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6de3480-97e0-4286-8f5b-ad67a7268bdf None admin] PATH_INFO: `/v3/services` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:19.010232 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc08779-6108-40aa-b2fd-3e5ea10594b1 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:19.010347 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6de3480-97e0-4286-8f5b-ad67a7268bdf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:19.012598 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6de3480-97e0-4286-8f5b-ad67a7268bdf None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:19.023674 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 399/785] 10.222.0.61 () {56 vars in 1087 bytes} [Mon Aug 22 19:06:18 2022] GET /identity/v3/services => generated 772 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 19:06:19.134648 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f048ea88-d228-40f1-b6e7-db81a08851dd None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:19.134980 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f048ea88-d228-40f1-b6e7-db81a08851dd None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:19.135194 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f048ea88-d228-40f1-b6e7-db81a08851dd None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:19.136875 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 400/786] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:06:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 19:06:19.149264 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f723d6c7-7677-4b6a-ac59-fd1acea376bd None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:19.149554 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f723d6c7-7677-4b6a-ac59-fd1acea376bd None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:19.149821 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f723d6c7-7677-4b6a-ac59-fd1acea376bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:19.181332 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc08779-6108-40aa-b2fd-3e5ea10594b1 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:19.183307 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:19.183531 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-f723d6c7-7677-4b6a-ac59-fd1acea376bd None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:06:19.184970 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-f723d6c7-7677-4b6a-ac59-fd1acea376bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:06:19.226357 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:19.228406 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5831b27-9e3e-4939-a0f5-0c4ad3f2c58d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:19.229120 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5831b27-9e3e-4939-a0f5-0c4ad3f2c58d None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:19.229704 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5831b27-9e3e-4939-a0f5-0c4ad3f2c58d None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:19.230007 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5831b27-9e3e-4939-a0f5-0c4ad3f2c58d None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:19.230633 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5831b27-9e3e-4939-a0f5-0c4ad3f2c58d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:19.231002 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5831b27-9e3e-4939-a0f5-0c4ad3f2c58d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:19.232420 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5831b27-9e3e-4939-a0f5-0c4ad3f2c58d None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:19.253831 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 387/787] 10.222.0.61 () {56 vars in 1109 bytes} [Mon Aug 22 19:06:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:19.260015 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5831b27-9e3e-4939-a0f5-0c4ad3f2c58d None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:19.276354 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 401/788] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:06:19 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 19:06:19.374226 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5831b27-9e3e-4939-a0f5-0c4ad3f2c58d None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:19.411871 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57673e95-8d0c-44b7-954d-26cf715f94fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:19.412867 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57673e95-8d0c-44b7-954d-26cf715f94fc None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:19.413384 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57673e95-8d0c-44b7-954d-26cf715f94fc None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:19.413851 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57673e95-8d0c-44b7-954d-26cf715f94fc None admin] PATH_INFO: `/v3/users/swift` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:19.421894 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57673e95-8d0c-44b7-954d-26cf715f94fc None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:19.423620 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57673e95-8d0c-44b7-954d-26cf715f94fc None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:19.435985 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-57673e95-8d0c-44b7-954d-26cf715f94fc None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 22 19:06:19.435985 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:19.435985 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:19.435985 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:19.435985 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:19.435985 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:19.435985 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:19.435985 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:19.435985 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:19.435985 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:19.435985 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:19.435985 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:19.435985 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 19:06:19.435985 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 19:06:19.435985 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 19:06:19.435985 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 19:06:19.438688 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:19.438688 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:19.438688 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 19:06:19.438688 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 19:06:19.438688 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 19:06:19.438688 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 19:06:19.438688 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:19.438688 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:19.438688 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:19.438688 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:19.438688 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:19.438688 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:19.438688 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:19.438688 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:19.438688 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:19.438688 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:19.438688 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:19.438688 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:19.439688 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:19.439688 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:19.439688 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 19:06:19.439688 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 19:06:19.439688 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 19:06:19.439688 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 19:06:19.439688 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 19:06:19.439688 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 19:06:19.439688 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 22 19:06:19.439688 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:06:19.442360 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 388/789] 10.222.0.61 () {56 vars in 1096 bytes} [Mon Aug 22 19:06:19 2022] GET /identity/v3/users/swift => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 19:06:19.447744 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f723d6c7-7677-4b6a-ac59-fd1acea376bd None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:19.596687 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f723d6c7-7677-4b6a-ac59-fd1acea376bd None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:19.641046 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d25f378-cb63-4e5d-89f9-44deb3a4c74f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:19.641660 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d25f378-cb63-4e5d-89f9-44deb3a4c74f None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:19.641939 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d25f378-cb63-4e5d-89f9-44deb3a4c74f None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:19.642212 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d25f378-cb63-4e5d-89f9-44deb3a4c74f None admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:19.642947 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d25f378-cb63-4e5d-89f9-44deb3a4c74f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:19.643251 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d25f378-cb63-4e5d-89f9-44deb3a4c74f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:19.644710 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d25f378-cb63-4e5d-89f9-44deb3a4c74f None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:19.667289 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 402/790] 10.222.0.61 () {56 vars in 1135 bytes} [Mon Aug 22 19:06:19 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:19.672853 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57673e95-8d0c-44b7-954d-26cf715f94fc None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:19.722260 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:19.736560 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d25f378-cb63-4e5d-89f9-44deb3a4c74f None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:19.766328 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:19.867276 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d25f378-cb63-4e5d-89f9-44deb3a4c74f None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:19.911363 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffe5bc5-88b7-4c78-8c07-9b2753e6c33c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:19.912401 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffe5bc5-88b7-4c78-8c07-9b2753e6c33c None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:19.912883 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffe5bc5-88b7-4c78-8c07-9b2753e6c33c None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:19.913376 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffe5bc5-88b7-4c78-8c07-9b2753e6c33c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:19.924992 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffe5bc5-88b7-4c78-8c07-9b2753e6c33c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 19:06:19.930042 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-5ffe5bc5-88b7-4c78-8c07-9b2753e6c33c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 19:06:19.930042 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:19.930042 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:19.930042 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:19.930042 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:19.930042 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:19.930042 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:19.930042 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:19.930042 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:19.930042 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:19.930042 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:19.930042 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:19.930042 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 19:06:19.930042 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 19:06:19.930042 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 19:06:19.930042 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 19:06:19.931005 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 19:06:19.931005 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 19:06:19.931005 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 19:06:19.931005 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 19:06:19.931005 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:19.931005 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:19.931005 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:19.931005 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:19.931005 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:19.931005 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:19.931005 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:19.931005 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:19.931005 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:19.931005 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:19.931005 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:19.931005 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:19.931882 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:19.931882 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:19.931882 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:19.931882 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:19.931882 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 19:06:19.931882 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 19:06:19.931882 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 19:06:19.931882 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 19:06:19.931882 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 19:06:19.931882 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 19:06:19.931882 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 19:06:19.931882 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:06:19.939557 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 403/791] 10.222.0.61 () {56 vars in 1102 bytes} [Mon Aug 22 19:06:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 19:06:19.941052 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60551644-db30-4e90-aa9b-406ba8db3b1c None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:19.941557 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60551644-db30-4e90-aa9b-406ba8db3b1c None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:19.942047 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60551644-db30-4e90-aa9b-406ba8db3b1c None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:19.943761 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 404/792] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:06:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:06:19.944705 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60551644-db30-4e90-aa9b-406ba8db3b1c None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:19.947218 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57673e95-8d0c-44b7-954d-26cf715f94fc None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:19.984503 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1739caaa-8be0-4162-b6ad-28d12964ab67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:19.985137 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1739caaa-8be0-4162-b6ad-28d12964ab67 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:19.985409 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1739caaa-8be0-4162-b6ad-28d12964ab67 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:19.985677 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1739caaa-8be0-4162-b6ad-28d12964ab67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:19.996710 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1739caaa-8be0-4162-b6ad-28d12964ab67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:19.998138 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1739caaa-8be0-4162-b6ad-28d12964ab67 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:20.013407 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-1739caaa-8be0-4162-b6ad-28d12964ab67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:06:20.013407 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:20.013407 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:20.013407 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:20.013407 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:20.013407 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:20.013407 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:20.013407 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:20.013407 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:20.013407 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:20.013407 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:20.013407 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:20.013407 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 19:06:20.013407 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 19:06:20.013407 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 19:06:20.013407 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 19:06:20.014372 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:20.014372 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:20.014372 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:20.014372 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:20.014372 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:20.014372 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:20.014372 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:20.014372 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:20.014372 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:20.014372 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:20.014372 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:20.014372 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:20.014372 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:20.014372 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:20.014372 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:20.014372 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:20.014372 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 19:06:20.015932 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 19:06:20.015932 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:06:20.015932 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:06:20.019753 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 389/793] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:06:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:20.021063 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaec5f8-f3a7-43ce-9aa5-085143ee2e96 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:20.021340 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaec5f8-f3a7-43ce-9aa5-085143ee2e96 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:20.021609 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaec5f8-f3a7-43ce-9aa5-085143ee2e96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:20.039857 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-3aaec5f8-f3a7-43ce-9aa5-085143ee2e96 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:06:20.040813 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-3aaec5f8-f3a7-43ce-9aa5-085143ee2e96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:06:20.043674 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:20.062301 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60551644-db30-4e90-aa9b-406ba8db3b1c None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:20.098747 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6940d77a-c015-4f80-a644-6df6c3c7ce2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:20.099720 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6940d77a-c015-4f80-a644-6df6c3c7ce2f None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:20.100206 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6940d77a-c015-4f80-a644-6df6c3c7ce2f None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:20.100724 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6940d77a-c015-4f80-a644-6df6c3c7ce2f None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:20.101597 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6940d77a-c015-4f80-a644-6df6c3c7ce2f None admin] RBAC: Adding query filter params (name=service) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:20.102111 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6940d77a-c015-4f80-a644-6df6c3c7ce2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:20.103744 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6940d77a-c015-4f80-a644-6df6c3c7ce2f None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:20.112832 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 390/794] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:06:20 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 19:06:20.113575 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 405/795] 10.222.0.61 () {56 vars in 1103 bytes} [Mon Aug 22 19:06:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:20.120816 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaec5f8-f3a7-43ce-9aa5-085143ee2e96 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:20.122836 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:20.125754 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6940d77a-c015-4f80-a644-6df6c3c7ce2f None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:20.235441 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaec5f8-f3a7-43ce-9aa5-085143ee2e96 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:20.301649 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432e3577-824e-4281-b6fd-962041b3eb6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:20.302322 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432e3577-824e-4281-b6fd-962041b3eb6a None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:20.302606 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432e3577-824e-4281-b6fd-962041b3eb6a None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:20.302877 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432e3577-824e-4281-b6fd-962041b3eb6a None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:20.303535 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432e3577-824e-4281-b6fd-962041b3eb6a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:20.303862 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432e3577-824e-4281-b6fd-962041b3eb6a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:20.305232 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432e3577-824e-4281-b6fd-962041b3eb6a None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:20.326809 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6940d77a-c015-4f80-a644-6df6c3c7ce2f None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:20.327391 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 391/796] 10.222.0.61 () {56 vars in 1109 bytes} [Mon Aug 22 19:06:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:20.332294 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432e3577-824e-4281-b6fd-962041b3eb6a None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:20.364569 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4974d7-92c1-4a28-a122-200e950c650b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:20.365509 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4974d7-92c1-4a28-a122-200e950c650b None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:20.365840 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4974d7-92c1-4a28-a122-200e950c650b None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:20.366120 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4974d7-92c1-4a28-a122-200e950c650b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:20.378951 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4974d7-92c1-4a28-a122-200e950c650b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:20.380580 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4974d7-92c1-4a28-a122-200e950c650b None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:20.393946 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:20.401815 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-9d4974d7-92c1-4a28-a122-200e950c650b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:06:20.401815 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:20.401815 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:20.401815 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:20.401815 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:20.401815 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:20.401815 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:20.401815 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:20.401815 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:20.401815 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:20.401815 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:20.401815 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:20.401815 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 19:06:20.401815 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 19:06:20.401815 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 19:06:20.401815 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 19:06:20.402842 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:20.402842 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:20.402842 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:20.402842 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:20.402842 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:20.402842 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:20.402842 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:20.402842 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:20.402842 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:20.402842 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:20.402842 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:20.402842 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:20.402842 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:20.402842 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:20.402842 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:20.402842 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:20.402842 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 19:06:20.403767 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 19:06:20.403767 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:06:20.403767 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:06:20.403767 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 406/797] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:06:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 19:06:20.409643 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4974d7-92c1-4a28-a122-200e950c650b None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:20.469153 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:20.515869 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432e3577-824e-4281-b6fd-962041b3eb6a None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:20.548952 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4974d7-92c1-4a28-a122-200e950c650b None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:20.558701 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79049d54-f28c-4b46-a79b-aeaacdffb07d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:20.559597 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79049d54-f28c-4b46-a79b-aeaacdffb07d None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:20.560435 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79049d54-f28c-4b46-a79b-aeaacdffb07d None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:20.560894 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79049d54-f28c-4b46-a79b-aeaacdffb07d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:20.569956 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79049d54-f28c-4b46-a79b-aeaacdffb07d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:20.571597 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79049d54-f28c-4b46-a79b-aeaacdffb07d None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:20.586349 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-79049d54-f28c-4b46-a79b-aeaacdffb07d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 19:06:20.586349 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:20.586349 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:20.586349 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:20.586349 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:20.586349 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:20.586349 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:20.586349 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:20.586349 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:20.586349 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:20.586349 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:20.586349 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:20.586349 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 19:06:20.586349 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 19:06:20.586349 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 19:06:20.587254 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 19:06:20.587254 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:20.587254 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:20.587254 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:20.587254 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:20.587254 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:20.587254 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:20.587254 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:20.587254 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:20.587254 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:20.587254 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:20.587254 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:20.587254 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:20.587254 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:20.587254 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:20.587254 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:20.587254 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:20.588199 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 19:06:20.588199 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 19:06:20.588199 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 19:06:20.588199 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 19:06:20.588199 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 19:06:20.588199 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 19:06:20.588199 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 19:06:20.588199 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:06:20.597406 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 392/798] 10.222.0.61 () {56 vars in 1111 bytes} [Mon Aug 22 19:06:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 19:06:20.597406 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79049d54-f28c-4b46-a79b-aeaacdffb07d None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:20.598043 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402cc653-111c-47fa-8a75-4cd50e336f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:20.598657 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402cc653-111c-47fa-8a75-4cd50e336f84 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:20.598935 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402cc653-111c-47fa-8a75-4cd50e336f84 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:20.599207 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402cc653-111c-47fa-8a75-4cd50e336f84 None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:20.599880 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402cc653-111c-47fa-8a75-4cd50e336f84 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:20.600185 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402cc653-111c-47fa-8a75-4cd50e336f84 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:20.601558 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402cc653-111c-47fa-8a75-4cd50e336f84 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:20.616212 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 407/799] 10.222.0.61 () {56 vars in 1109 bytes} [Mon Aug 22 19:06:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 19:06:20.617343 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402cc653-111c-47fa-8a75-4cd50e336f84 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:20.680778 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:20.717106 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79049d54-f28c-4b46-a79b-aeaacdffb07d None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:20.726439 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402cc653-111c-47fa-8a75-4cd50e336f84 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:20.746190 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:20.758115 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92ddc3c-ebfa-4c91-998b-0307f8a5a33f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:20.758708 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92ddc3c-ebfa-4c91-998b-0307f8a5a33f None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:20.758991 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92ddc3c-ebfa-4c91-998b-0307f8a5a33f None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:20.759266 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92ddc3c-ebfa-4c91-998b-0307f8a5a33f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:20.766597 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f549868-bc57-4d61-ad2b-4420eba3ed1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:20.767527 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f549868-bc57-4d61-ad2b-4420eba3ed1d None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:20.768042 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f549868-bc57-4d61-ad2b-4420eba3ed1d None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:20.768537 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f549868-bc57-4d61-ad2b-4420eba3ed1d None admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:20.769435 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f549868-bc57-4d61-ad2b-4420eba3ed1d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:20.769941 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f549868-bc57-4d61-ad2b-4420eba3ed1d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:20.771581 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f549868-bc57-4d61-ad2b-4420eba3ed1d None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:20.771832 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92ddc3c-ebfa-4c91-998b-0307f8a5a33f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 19:06:20.776885 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-f92ddc3c-ebfa-4c91-998b-0307f8a5a33f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 19:06:20.776885 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:20.776885 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:20.776885 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:20.776885 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:20.776885 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:20.776885 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:20.776885 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:20.776885 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:20.776885 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:20.776885 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:20.776885 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:20.776885 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 19:06:20.776885 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 19:06:20.776885 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 19:06:20.776885 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 19:06:20.777832 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 19:06:20.777832 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 19:06:20.777832 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 19:06:20.777832 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 19:06:20.777832 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:20.777832 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:20.777832 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:20.777832 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:20.777832 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:20.777832 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:20.777832 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:20.777832 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:20.777832 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:20.777832 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:20.777832 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:20.777832 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:20.778688 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:20.778688 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:20.778688 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:20.778688 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:20.778688 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 19:06:20.778688 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 19:06:20.778688 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 19:06:20.778688 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 19:06:20.778688 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 19:06:20.778688 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 19:06:20.778688 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 19:06:20.778688 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:06:20.778688 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 393/800] 10.222.0.61 () {56 vars in 1096 bytes} [Mon Aug 22 19:06:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:20.779382 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92ddc3c-ebfa-4c91-998b-0307f8a5a33f None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:20.795726 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 408/801] 10.222.0.61 () {56 vars in 1148 bytes} [Mon Aug 22 19:06:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:20.797531 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f549868-bc57-4d61-ad2b-4420eba3ed1d None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:20.896985 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92ddc3c-ebfa-4c91-998b-0307f8a5a33f None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:20.904356 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f549868-bc57-4d61-ad2b-4420eba3ed1d None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:20.926048 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:20.938338 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8febf955-b068-4c4e-b661-6084aa360970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:20.939384 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8febf955-b068-4c4e-b661-6084aa360970 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:20.940034 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8febf955-b068-4c4e-b661-6084aa360970 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:20.940457 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8febf955-b068-4c4e-b661-6084aa360970 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:20.942213 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3d7f63-0999-412a-9d02-12c7ccff6a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:20.942934 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3d7f63-0999-412a-9d02-12c7ccff6a81 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:20.943316 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3d7f63-0999-412a-9d02-12c7ccff6a81 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:20.943706 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3d7f63-0999-412a-9d02-12c7ccff6a81 None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:20.944689 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3d7f63-0999-412a-9d02-12c7ccff6a81 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:20.944862 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3d7f63-0999-412a-9d02-12c7ccff6a81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:20.946329 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3d7f63-0999-412a-9d02-12c7ccff6a81 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:20.950308 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8febf955-b068-4c4e-b661-6084aa360970 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:20.952974 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8febf955-b068-4c4e-b661-6084aa360970 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:20.956844 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 409/802] 10.222.0.61 () {56 vars in 1099 bytes} [Mon Aug 22 19:06:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:20.957811 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3d7f63-0999-412a-9d02-12c7ccff6a81 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:20.966468 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-8febf955-b068-4c4e-b661-6084aa360970 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 22 19:06:20.966468 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:20.966468 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:20.966468 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:20.966468 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:20.966468 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:20.966468 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:20.966468 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:20.966468 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:20.966468 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:20.966468 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:20.966468 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:20.966468 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 19:06:20.966468 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 19:06:20.966468 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 19:06:20.966468 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 19:06:20.967469 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:20.967469 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:20.967469 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 19:06:20.967469 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 19:06:20.967469 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 19:06:20.967469 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 19:06:20.967469 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:20.967469 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:20.967469 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:20.967469 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:20.967469 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:20.967469 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:20.967469 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:20.967469 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:20.967469 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:20.967469 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:20.967469 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:20.967469 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:20.969071 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:20.969071 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:20.969071 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 19:06:20.969071 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 19:06:20.969071 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 19:06:20.969071 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 19:06:20.969071 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 19:06:20.969071 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 19:06:20.969071 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 22 19:06:20.969071 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:06:20.975779 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 394/803] 10.222.0.61 () {56 vars in 1117 bytes} [Mon Aug 22 19:06:20 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:20.977272 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8febf955-b068-4c4e-b661-6084aa360970 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:21.032049 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:21.064974 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3d7f63-0999-412a-9d02-12c7ccff6a81 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:21.097652 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8febf955-b068-4c4e-b661-6084aa360970 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:21.100896 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93bbad8-25ad-46f2-a8af-2b88cca5f991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:21.101563 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93bbad8-25ad-46f2-a8af-2b88cca5f991 None admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:21.101845 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93bbad8-25ad-46f2-a8af-2b88cca5f991 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:21.102112 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93bbad8-25ad-46f2-a8af-2b88cca5f991 None admin] PATH_INFO: `/v3/projects/e07dc5ebdc504c6da706a2d21c0a3970/users/2824359491934d0dacaa21a718becee5/roles/163c30c3b3c246beb849ce176f02d9da` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:21.104100 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93bbad8-25ad-46f2-a8af-2b88cca5f991 None admin] RBAC: Authorizing `identity:create_grant(project_id=e07dc5ebdc504c6da706a2d21c0a3970, user_id=2824359491934d0dacaa21a718becee5, role_id=163c30c3b3c246beb849ce176f02d9da)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:21.105515 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93bbad8-25ad-46f2-a8af-2b88cca5f991 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:21.127755 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 410/804] 10.222.0.61 () {58 vars in 1439 bytes} [Mon Aug 22 19:06:20 2022] PUT /identity/v3/projects/e07dc5ebdc504c6da706a2d21c0a3970/users/2824359491934d0dacaa21a718becee5/roles/163c30c3b3c246beb849ce176f02d9da => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:06:21.129975 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93bbad8-25ad-46f2-a8af-2b88cca5f991 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:21.142490 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe50480-b991-4bb1-8981-7e71c3e4102a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:21.143402 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe50480-b991-4bb1-8981-7e71c3e4102a None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:21.143814 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe50480-b991-4bb1-8981-7e71c3e4102a None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:21.147389 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe50480-b991-4bb1-8981-7e71c3e4102a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:21.161503 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe50480-b991-4bb1-8981-7e71c3e4102a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:21.163207 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe50480-b991-4bb1-8981-7e71c3e4102a None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:21.183613 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-1fe50480-b991-4bb1-8981-7e71c3e4102a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:06:21.183613 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:21.183613 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:21.183613 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:21.183613 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:21.183613 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:21.183613 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:21.183613 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:21.183613 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:21.183613 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:21.183613 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:21.183613 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:21.183613 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 19:06:21.183613 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 19:06:21.183613 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 19:06:21.183613 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 19:06:21.185600 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:21.185600 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:21.185600 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:21.185600 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:21.185600 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:21.185600 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:21.185600 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:21.185600 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:21.185600 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:21.185600 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:21.185600 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:21.185600 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:21.185600 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:21.185600 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:21.185600 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:21.185600 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:21.185600 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 19:06:21.189475 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 19:06:21.189475 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:06:21.189475 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:06:21.190871 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 395/805] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:06:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:21.192550 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c50b081-94fb-42df-93f4-a2ce949e4ac0 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:21.192866 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c50b081-94fb-42df-93f4-a2ce949e4ac0 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:21.193211 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c50b081-94fb-42df-93f4-a2ce949e4ac0 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:21.198781 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 396/806] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:06:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:06:21.198781 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c50b081-94fb-42df-93f4-a2ce949e4ac0 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:21.220901 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:21.300141 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93bbad8-25ad-46f2-a8af-2b88cca5f991 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:21.324084 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:21.329613 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c50b081-94fb-42df-93f4-a2ce949e4ac0 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:21.340219 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3abcfd7-747c-40ff-9dbc-b511276d88ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:21.341163 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3abcfd7-747c-40ff-9dbc-b511276d88ec None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:21.341458 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3abcfd7-747c-40ff-9dbc-b511276d88ec None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:21.341927 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3abcfd7-747c-40ff-9dbc-b511276d88ec None admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:21.342815 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3abcfd7-747c-40ff-9dbc-b511276d88ec None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:21.343675 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3abcfd7-747c-40ff-9dbc-b511276d88ec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:21.345472 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3abcfd7-747c-40ff-9dbc-b511276d88ec None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:21.370297 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb9aa8f-a74d-4173-ac4d-5336ce20e612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:21.370536 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 411/807] 10.222.0.61 () {56 vars in 1149 bytes} [Mon Aug 22 19:06:21 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:21.370639 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb9aa8f-a74d-4173-ac4d-5336ce20e612 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:21.370639 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb9aa8f-a74d-4173-ac4d-5336ce20e612 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:21.370639 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb9aa8f-a74d-4173-ac4d-5336ce20e612 None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:21.370879 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e469c5-e9ff-4bc7-b21d-97459da469d6 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:21.370879 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e469c5-e9ff-4bc7-b21d-97459da469d6 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:21.371009 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb9aa8f-a74d-4173-ac4d-5336ce20e612 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:21.371085 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e469c5-e9ff-4bc7-b21d-97459da469d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:21.371166 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb9aa8f-a74d-4173-ac4d-5336ce20e612 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:21.371166 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb9aa8f-a74d-4173-ac4d-5336ce20e612 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:21.385847 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 397/808] 10.222.0.61 () {56 vars in 1109 bytes} [Mon Aug 22 19:06:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:21.385847 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c252f06-17d6-4d08-82a0-4fa7865b4a01 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:21.385847 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c252f06-17d6-4d08-82a0-4fa7865b4a01 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:21.385847 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c252f06-17d6-4d08-82a0-4fa7865b4a01 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:21.388926 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 398/809] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:06:21 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:06:21.389586 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c252f06-17d6-4d08-82a0-4fa7865b4a01 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:21.393010 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-f4e469c5-e9ff-4bc7-b21d-97459da469d6 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:06:21.394299 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-f4e469c5-e9ff-4bc7-b21d-97459da469d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:06:21.471966 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 412/810] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:06:21 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 19:06:21.471966 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e469c5-e9ff-4bc7-b21d-97459da469d6 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:21.507987 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c252f06-17d6-4d08-82a0-4fa7865b4a01 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:21.515732 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:21.548221 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231546a0-39f3-4930-af6e-a6ba792e94df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:21.548882 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231546a0-39f3-4930-af6e-a6ba792e94df None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:21.549147 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231546a0-39f3-4930-af6e-a6ba792e94df None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:21.549420 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231546a0-39f3-4930-af6e-a6ba792e94df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:21.560700 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231546a0-39f3-4930-af6e-a6ba792e94df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:21.562179 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231546a0-39f3-4930-af6e-a6ba792e94df None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:21.579778 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-231546a0-39f3-4930-af6e-a6ba792e94df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:06:21.579778 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:21.579778 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:21.579778 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:21.579778 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:21.579778 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:21.579778 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:21.579778 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:21.579778 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:21.579778 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:21.579778 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:21.579778 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:21.579778 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 19:06:21.579778 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 19:06:21.579778 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 19:06:21.579778 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 19:06:21.580852 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:21.580852 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:21.580852 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:21.580852 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:21.580852 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:21.580852 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:21.580852 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:21.580852 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:21.580852 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:21.580852 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:21.580852 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:21.580852 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:21.580852 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:21.580852 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:21.580852 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:21.580852 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:21.580852 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 19:06:21.582139 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 19:06:21.582139 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:06:21.582139 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:06:21.582139 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 399/811] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:06:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:21.582373 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82570bb-25e7-4149-81cd-31369b51d3d4 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:21.582635 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82570bb-25e7-4149-81cd-31369b51d3d4 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:21.583806 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82570bb-25e7-4149-81cd-31369b51d3d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:21.604167 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-c82570bb-25e7-4149-81cd-31369b51d3d4 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:06:21.605360 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-c82570bb-25e7-4149-81cd-31369b51d3d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:06:21.612937 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e469c5-e9ff-4bc7-b21d-97459da469d6 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:21.615980 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:21.649568 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc68aac2-1f8c-467f-bc7a-02dc2ef132b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:21.650568 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc68aac2-1f8c-467f-bc7a-02dc2ef132b5 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:21.651048 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc68aac2-1f8c-467f-bc7a-02dc2ef132b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:21.651517 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc68aac2-1f8c-467f-bc7a-02dc2ef132b5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:21.659592 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc68aac2-1f8c-467f-bc7a-02dc2ef132b5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:21.661731 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc68aac2-1f8c-467f-bc7a-02dc2ef132b5 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:21.673600 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-cc68aac2-1f8c-467f-bc7a-02dc2ef132b5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 22 19:06:21.673600 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:21.673600 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:21.673600 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:21.673600 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:21.673600 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:21.673600 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:21.673600 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:21.673600 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:21.673600 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:21.673600 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:21.673600 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:21.673600 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 19:06:21.673600 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 19:06:21.673600 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 19:06:21.673600 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 19:06:21.674545 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:21.674545 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:21.674545 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 19:06:21.674545 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 19:06:21.674545 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 19:06:21.674545 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 19:06:21.674545 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:21.674545 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:21.674545 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:21.674545 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:21.674545 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:21.674545 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:21.674545 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:21.674545 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:21.674545 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:21.674545 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:21.674545 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:21.674545 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:21.675661 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:21.675661 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:21.675661 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 19:06:21.675661 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 19:06:21.675661 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 19:06:21.675661 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 19:06:21.675661 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 19:06:21.675661 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 19:06:21.675661 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 22 19:06:21.675661 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:06:21.682768 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 413/812] 10.222.0.61 () {56 vars in 1093 bytes} [Mon Aug 22 19:06:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:21.683922 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc68aac2-1f8c-467f-bc7a-02dc2ef132b5 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:21.684030 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 400/813] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:06:21 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 19:06:21.685906 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82570bb-25e7-4149-81cd-31369b51d3d4 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:21.808909 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:21.827514 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc68aac2-1f8c-467f-bc7a-02dc2ef132b5 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:21.844776 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82570bb-25e7-4149-81cd-31369b51d3d4 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:21.865228 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b20f5e-ea94-4bd4-a253-c985f99f64e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:21.866007 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b20f5e-ea94-4bd4-a253-c985f99f64e5 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:21.866382 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b20f5e-ea94-4bd4-a253-c985f99f64e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:21.866768 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b20f5e-ea94-4bd4-a253-c985f99f64e5 None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:21.867513 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b20f5e-ea94-4bd4-a253-c985f99f64e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:21.867935 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b20f5e-ea94-4bd4-a253-c985f99f64e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:21.869508 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b20f5e-ea94-4bd4-a253-c985f99f64e5 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:21.882429 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 414/814] 10.222.0.61 () {56 vars in 1109 bytes} [Mon Aug 22 19:06:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:21.887275 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b20f5e-ea94-4bd4-a253-c985f99f64e5 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:21.891380 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d4acc8-7421-4219-b715-11a63487068f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:21.892429 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d4acc8-7421-4219-b715-11a63487068f None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:21.893742 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d4acc8-7421-4219-b715-11a63487068f None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:21.894172 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d4acc8-7421-4219-b715-11a63487068f None admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:21.895054 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d4acc8-7421-4219-b715-11a63487068f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:21.895561 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d4acc8-7421-4219-b715-11a63487068f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:21.897136 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d4acc8-7421-4219-b715-11a63487068f None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:21.909378 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:21.927716 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 401/815] 10.222.0.61 () {56 vars in 1133 bytes} [Mon Aug 22 19:06:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:21.928764 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d4acc8-7421-4219-b715-11a63487068f None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:22.003620 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b20f5e-ea94-4bd4-a253-c985f99f64e5 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:22.043944 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24dd425-17a5-414f-8812-e66b3eb9bd32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:22.044818 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24dd425-17a5-414f-8812-e66b3eb9bd32 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:22.044937 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d4acc8-7421-4219-b715-11a63487068f None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:22.045681 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24dd425-17a5-414f-8812-e66b3eb9bd32 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:22.046126 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24dd425-17a5-414f-8812-e66b3eb9bd32 None admin] PATH_INFO: `/v3/services/network` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:22.047806 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24dd425-17a5-414f-8812-e66b3eb9bd32 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:22.049239 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24dd425-17a5-414f-8812-e66b3eb9bd32 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:22.058180 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-c24dd425-17a5-414f-8812-e66b3eb9bd32 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 22 19:06:22.058180 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:22.058180 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 19:06:22.058180 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 19:06:22.058180 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 19:06:22.058180 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 19:06:22.058180 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 19:06:22.058180 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 19:06:22.058180 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 22 19:06:22.058180 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:06:22.058180 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 19:06:22.058180 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:06:22.058180 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:22.058180 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:22.058180 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:22.058180 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:22.058180 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:22.059240 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:22.059240 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:22.059240 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:22.059240 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:22.059240 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:22.059240 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:22.059240 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 19:06:22.059240 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 19:06:22.059240 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 19:06:22.059240 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 19:06:22.059240 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:22.059240 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:22.059240 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:22.059240 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:22.059240 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:22.059240 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:22.060144 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:22.060144 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:22.060144 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:22.060144 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:22.060144 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:22.060144 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:22.060144 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:22.060144 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:22.060144 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:22.060144 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:22.060144 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 19:06:22.060144 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 19:06:22.060144 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 22 19:06:22.060144 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:06:22.060144 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 415/816] 10.222.0.61 () {56 vars in 1111 bytes} [Mon Aug 22 19:06:21 2022] GET /identity/v3/services/network => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:22.064733 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24dd425-17a5-414f-8812-e66b3eb9bd32 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:22.083625 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fd273a-1d58-462a-8ed3-dd5df2b46d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:22.084400 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fd273a-1d58-462a-8ed3-dd5df2b46d97 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:22.084583 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fd273a-1d58-462a-8ed3-dd5df2b46d97 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:22.084857 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fd273a-1d58-462a-8ed3-dd5df2b46d97 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:22.086111 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:22.092428 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fd273a-1d58-462a-8ed3-dd5df2b46d97 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:22.094037 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fd273a-1d58-462a-8ed3-dd5df2b46d97 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:22.105234 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-14fd273a-1d58-462a-8ed3-dd5df2b46d97 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 19:06:22.105234 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:22.105234 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:22.105234 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:22.105234 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:22.105234 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:22.105234 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:22.105234 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:22.105234 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:22.105234 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:22.105234 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:22.105234 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:22.105234 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 19:06:22.105234 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 19:06:22.105234 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 19:06:22.106157 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 19:06:22.106157 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:22.106157 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:22.106157 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:22.106157 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:22.106157 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:22.106157 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:22.106157 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:22.106157 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:22.106157 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:22.106157 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:22.106157 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:22.106157 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:22.106157 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:22.106157 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:22.106157 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:22.106157 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:22.107109 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 19:06:22.107109 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 19:06:22.107109 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 19:06:22.107109 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 19:06:22.107109 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 19:06:22.107109 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 19:06:22.107109 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 19:06:22.107109 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:06:22.107109 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 402/817] 10.222.0.61 () {56 vars in 1111 bytes} [Mon Aug 22 19:06:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:22.107109 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fd273a-1d58-462a-8ed3-dd5df2b46d97 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:22.174860 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:22.187746 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24dd425-17a5-414f-8812-e66b3eb9bd32 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:22.221699 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fd273a-1d58-462a-8ed3-dd5df2b46d97 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:22.228789 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b53396-9746-44e3-b778-44a944d90455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:22.229470 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b53396-9746-44e3-b778-44a944d90455 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:22.229751 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b53396-9746-44e3-b778-44a944d90455 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:22.230018 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b53396-9746-44e3-b778-44a944d90455 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:22.258099 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3eeba0-68f5-4332-bf92-f0cf885d6188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:22.258921 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3eeba0-68f5-4332-bf92-f0cf885d6188 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:22.259279 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3eeba0-68f5-4332-bf92-f0cf885d6188 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:22.259618 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3eeba0-68f5-4332-bf92-f0cf885d6188 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:22.260300 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3eeba0-68f5-4332-bf92-f0cf885d6188 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:22.261963 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3eeba0-68f5-4332-bf92-f0cf885d6188 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:22.271602 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-fc3eeba0-68f5-4332-bf92-f0cf885d6188 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 22 19:06:22.271602 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:22.271602 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 19:06:22.271602 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 19:06:22.271602 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 19:06:22.271602 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 19:06:22.271602 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 19:06:22.271602 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 19:06:22.271602 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 22 19:06:22.271602 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:06:22.271602 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 19:06:22.271602 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:06:22.271602 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:22.271602 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:22.271602 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:22.271602 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:22.271602 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:22.272845 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:22.272845 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:22.272845 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:22.272845 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:22.272845 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:22.272845 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:22.272845 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 19:06:22.272845 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 19:06:22.272845 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 19:06:22.272845 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 19:06:22.272845 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:22.272845 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:22.272845 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:22.272845 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:22.272845 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:22.272845 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:22.273826 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:22.273826 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:22.273826 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:22.273826 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:22.273826 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:22.273826 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:22.273826 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:22.273826 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:22.273826 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:22.273826 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:22.273826 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 19:06:22.273826 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 19:06:22.273826 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 22 19:06:22.273826 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:06:22.273826 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 403/818] 10.222.0.61 () {56 vars in 1129 bytes} [Mon Aug 22 19:06:22 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:22.276302 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3eeba0-68f5-4332-bf92-f0cf885d6188 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:22.305881 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b53396-9746-44e3-b778-44a944d90455 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:22.307452 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b53396-9746-44e3-b778-44a944d90455 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:22.323096 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:22.381299 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3eeba0-68f5-4332-bf92-f0cf885d6188 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:22.404506 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:22.423670 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 416/819] 10.222.0.61 () {58 vars in 1282 bytes} [Mon Aug 22 19:06:22 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 22 19:06:22.423879 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775418d0-7060-4af5-bd48-27e44542796f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:22.423879 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775418d0-7060-4af5-bd48-27e44542796f None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:22.424065 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b53396-9746-44e3-b778-44a944d90455 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:22.424155 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775418d0-7060-4af5-bd48-27e44542796f None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:22.424155 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775418d0-7060-4af5-bd48-27e44542796f None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:22.424155 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775418d0-7060-4af5-bd48-27e44542796f None admin] RBAC: Adding query filter params (name=network) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:22.424155 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775418d0-7060-4af5-bd48-27e44542796f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:22.424155 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775418d0-7060-4af5-bd48-27e44542796f None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:22.427684 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 404/820] 10.222.0.61 () {56 vars in 1112 bytes} [Mon Aug 22 19:06:22 2022] GET /identity/v3/services?name=network => generated 124 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:22.427684 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775418d0-7060-4af5-bd48-27e44542796f None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:22.529806 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b53396-9746-44e3-b778-44a944d90455 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:22.532253 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775418d0-7060-4af5-bd48-27e44542796f None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:22.544205 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:22.567767 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1769f79f-bf0c-4654-adce-8cd223a5a2a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:22.568372 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1769f79f-bf0c-4654-adce-8cd223a5a2a1 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:22.568647 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1769f79f-bf0c-4654-adce-8cd223a5a2a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:22.568750 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f211c281-be9c-4a1b-9d33-d8649003d433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:22.568922 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1769f79f-bf0c-4654-adce-8cd223a5a2a1 None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:22.569291 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f211c281-be9c-4a1b-9d33-d8649003d433 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:22.569539 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1769f79f-bf0c-4654-adce-8cd223a5a2a1 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:22.569647 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f211c281-be9c-4a1b-9d33-d8649003d433 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:22.569827 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f211c281-be9c-4a1b-9d33-d8649003d433 None admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:22.569933 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1769f79f-bf0c-4654-adce-8cd223a5a2a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:22.570518 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f211c281-be9c-4a1b-9d33-d8649003d433 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:22.570824 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f211c281-be9c-4a1b-9d33-d8649003d433 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:22.571172 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1769f79f-bf0c-4654-adce-8cd223a5a2a1 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:22.572449 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f211c281-be9c-4a1b-9d33-d8649003d433 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:22.580253 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 405/821] 10.222.0.61 () {56 vars in 1124 bytes} [Mon Aug 22 19:06:22 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:22.580253 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1769f79f-bf0c-4654-adce-8cd223a5a2a1 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:22.586152 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 417/822] 10.222.0.61 () {56 vars in 1148 bytes} [Mon Aug 22 19:06:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:22.591598 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f211c281-be9c-4a1b-9d33-d8649003d433 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:22.633405 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:22.688046 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1769f79f-bf0c-4654-adce-8cd223a5a2a1 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:22.703593 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f211c281-be9c-4a1b-9d33-d8649003d433 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:22.724203 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e91855-9349-4bcc-8c86-19defaca333b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:22.724895 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e91855-9349-4bcc-8c86-19defaca333b None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:22.725173 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e91855-9349-4bcc-8c86-19defaca333b None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:22.725439 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e91855-9349-4bcc-8c86-19defaca333b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:22.736047 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e91855-9349-4bcc-8c86-19defaca333b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:22.737477 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e91855-9349-4bcc-8c86-19defaca333b None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:22.744094 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd90b42-c06b-424a-974f-51f6965ab4d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:22.745179 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd90b42-c06b-424a-974f-51f6965ab4d2 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:22.745558 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd90b42-c06b-424a-974f-51f6965ab4d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:22.746312 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd90b42-c06b-424a-974f-51f6965ab4d2 None admin] PATH_INFO: `/v3/services` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:22.747348 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd90b42-c06b-424a-974f-51f6965ab4d2 None admin] RBAC: Adding query filter params (type=network) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:22.747778 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd90b42-c06b-424a-974f-51f6965ab4d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:22.749306 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd90b42-c06b-424a-974f-51f6965ab4d2 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:22.752021 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-51e91855-9349-4bcc-8c86-19defaca333b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:06:22.752021 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:22.752021 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:22.752021 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:22.752021 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:22.752021 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:22.752021 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:22.752021 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:22.752021 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:22.752021 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:22.752021 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:22.752021 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:22.752021 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 19:06:22.752021 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 19:06:22.752021 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 19:06:22.752021 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 19:06:22.753111 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:22.753111 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:22.753111 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:22.753111 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:22.753111 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:22.753111 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:22.753111 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:22.753111 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:22.753111 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:22.753111 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:22.753111 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:22.753111 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:22.753111 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:22.753111 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:22.753111 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:22.753111 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:22.753111 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 19:06:22.758693 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 19:06:22.758693 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:06:22.758693 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:06:22.758693 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 406/823] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:06:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:22.758693 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e91855-9349-4bcc-8c86-19defaca333b None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:22.765690 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 418/824] 10.222.0.61 () {56 vars in 1112 bytes} [Mon Aug 22 19:06:22 2022] GET /identity/v3/services?type=network => generated 350 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:22.765690 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd90b42-c06b-424a-974f-51f6965ab4d2 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:22.806646 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:22.858523 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e91855-9349-4bcc-8c86-19defaca333b None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:22.860472 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:22.875348 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd90b42-c06b-424a-974f-51f6965ab4d2 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:22.893409 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32562892-4bbe-4a9d-b725-631bdd91001a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:22.894028 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32562892-4bbe-4a9d-b725-631bdd91001a None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:22.894307 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32562892-4bbe-4a9d-b725-631bdd91001a None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:22.894578 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32562892-4bbe-4a9d-b725-631bdd91001a None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:22.895242 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32562892-4bbe-4a9d-b725-631bdd91001a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:22.895545 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32562892-4bbe-4a9d-b725-631bdd91001a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:22.896928 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32562892-4bbe-4a9d-b725-631bdd91001a None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:22.901650 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 407/825] 10.222.0.61 () {56 vars in 1124 bytes} [Mon Aug 22 19:06:22 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:22.902423 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32562892-4bbe-4a9d-b725-631bdd91001a None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:22.917723 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f3d447-4191-4a20-a7e9-e741713252c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:22.918646 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f3d447-4191-4a20-a7e9-e741713252c2 None admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:22.919068 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f3d447-4191-4a20-a7e9-e741713252c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:22.919483 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f3d447-4191-4a20-a7e9-e741713252c2 None admin] PATH_INFO: `/v3/projects/e07dc5ebdc504c6da706a2d21c0a3970/users/399396a8fe254da199bc48b0ca5243d0/roles/324bd04d55c34b61a5282c9f6ed73f46` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:22.937610 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f3d447-4191-4a20-a7e9-e741713252c2 None admin] RBAC: Authorizing `identity:create_grant(project_id=e07dc5ebdc504c6da706a2d21c0a3970, user_id=399396a8fe254da199bc48b0ca5243d0, role_id=324bd04d55c34b61a5282c9f6ed73f46)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:22.939117 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f3d447-4191-4a20-a7e9-e741713252c2 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:22.953101 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 419/826] 10.222.0.61 () {58 vars in 1439 bytes} [Mon Aug 22 19:06:22 2022] PUT /identity/v3/projects/e07dc5ebdc504c6da706a2d21c0a3970/users/399396a8fe254da199bc48b0ca5243d0/roles/324bd04d55c34b61a5282c9f6ed73f46 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:06:22.954027 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f3d447-4191-4a20-a7e9-e741713252c2 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:23.038143 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32562892-4bbe-4a9d-b725-631bdd91001a None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:23.056258 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:23.075811 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f3d447-4191-4a20-a7e9-e741713252c2 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:23.078705 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5884f6-20d2-4c6d-a9e2-e7dac67a7d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:23.079367 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5884f6-20d2-4c6d-a9e2-e7dac67a7d5c None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:23.079669 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5884f6-20d2-4c6d-a9e2-e7dac67a7d5c None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:23.079971 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5884f6-20d2-4c6d-a9e2-e7dac67a7d5c None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:23.080669 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5884f6-20d2-4c6d-a9e2-e7dac67a7d5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:23.080974 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5884f6-20d2-4c6d-a9e2-e7dac67a7d5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:23.082432 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5884f6-20d2-4c6d-a9e2-e7dac67a7d5c None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:23.101535 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 408/827] 10.222.0.61 () {56 vars in 1109 bytes} [Mon Aug 22 19:06:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:23.101535 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5884f6-20d2-4c6d-a9e2-e7dac67a7d5c None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:23.126018 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd68480-16e0-4ff7-a2f0-d52509930a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:23.126531 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd68480-16e0-4ff7-a2f0-d52509930a16 None admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:23.126807 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd68480-16e0-4ff7-a2f0-d52509930a16 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:23.127074 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd68480-16e0-4ff7-a2f0-d52509930a16 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:23.127796 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd68480-16e0-4ff7-a2f0-d52509930a16 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:23.129158 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd68480-16e0-4ff7-a2f0-d52509930a16 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:23.134709 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:23.159690 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 420/828] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 19:06:22 2022] POST /identity/v3/endpoints => generated 337 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:06:23.159690 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd4b85d-ff1a-4aed-9fad-8fda164faf4c None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:23.159690 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd4b85d-ff1a-4aed-9fad-8fda164faf4c None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:23.159690 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd4b85d-ff1a-4aed-9fad-8fda164faf4c None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:23.159690 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 421/829] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:06:23 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:06:23.160194 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd4b85d-ff1a-4aed-9fad-8fda164faf4c None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:23.247965 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5884f6-20d2-4c6d-a9e2-e7dac67a7d5c None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:23.297872 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6699369-1aee-4479-bf2d-fa98a7aeb466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:23.298497 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6699369-1aee-4479-bf2d-fa98a7aeb466 None admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:23.298777 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6699369-1aee-4479-bf2d-fa98a7aeb466 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:23.299051 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6699369-1aee-4479-bf2d-fa98a7aeb466 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:23.300630 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6699369-1aee-4479-bf2d-fa98a7aeb466 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:23.301351 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6699369-1aee-4479-bf2d-fa98a7aeb466 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:23.317992 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd4b85d-ff1a-4aed-9fad-8fda164faf4c None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:23.330495 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 409/830] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 19:06:23 2022] POST /identity/v3/endpoints => generated 346 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:06:23.332137 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2128d09-e287-4f19-8a4b-71f9a3c33aed None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:23.332566 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2128d09-e287-4f19-8a4b-71f9a3c33aed None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:23.333065 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2128d09-e287-4f19-8a4b-71f9a3c33aed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:23.355523 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-d2128d09-e287-4f19-8a4b-71f9a3c33aed None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:06:23.357419 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-d2128d09-e287-4f19-8a4b-71f9a3c33aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:06:23.371581 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f2a688-f1fc-47a5-bd10-415edf4d0cb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:23.372708 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f2a688-f1fc-47a5-bd10-415edf4d0cb3 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:23.372708 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f2a688-f1fc-47a5-bd10-415edf4d0cb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:23.373124 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f2a688-f1fc-47a5-bd10-415edf4d0cb3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:23.381541 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f2a688-f1fc-47a5-bd10-415edf4d0cb3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:23.383089 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f2a688-f1fc-47a5-bd10-415edf4d0cb3 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:23.388282 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:23.400363 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-f9f2a688-f1fc-47a5-bd10-415edf4d0cb3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 19:06:23.400363 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:23.400363 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:23.400363 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:23.400363 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:23.400363 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:23.400363 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:23.400363 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:23.400363 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:23.400363 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:23.400363 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:23.400363 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:23.400363 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 19:06:23.400363 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 19:06:23.400363 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 19:06:23.401642 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 19:06:23.401642 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:23.401642 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:23.401642 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:23.401642 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:23.401642 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:23.401642 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:23.401642 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:23.401642 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:23.401642 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:23.401642 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:23.401642 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:23.401642 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:23.401642 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:23.401642 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:23.401642 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:23.401642 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:23.402908 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 19:06:23.402908 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 19:06:23.402908 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 19:06:23.402908 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 19:06:23.402908 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 19:06:23.402908 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 19:06:23.402908 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 19:06:23.402908 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:06:23.409407 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 422/831] 10.222.0.61 () {56 vars in 1111 bytes} [Mon Aug 22 19:06:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:23.411174 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f2a688-f1fc-47a5-bd10-415edf4d0cb3 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:23.461384 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 410/832] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:06:23 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 19:06:23.531338 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f2a688-f1fc-47a5-bd10-415edf4d0cb3 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:23.566858 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b81b83-0828-4388-b79a-9c3d908d40de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:23.567558 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b81b83-0828-4388-b79a-9c3d908d40de None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:23.567943 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b81b83-0828-4388-b79a-9c3d908d40de None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:23.568281 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b81b83-0828-4388-b79a-9c3d908d40de None admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:23.569045 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b81b83-0828-4388-b79a-9c3d908d40de None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:23.569422 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b81b83-0828-4388-b79a-9c3d908d40de None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:23.570912 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b81b83-0828-4388-b79a-9c3d908d40de None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:23.583240 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 423/833] 10.222.0.61 () {56 vars in 1148 bytes} [Mon Aug 22 19:06:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 19:06:23.589239 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2128d09-e287-4f19-8a4b-71f9a3c33aed None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:23.600741 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:23.699797 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2128d09-e287-4f19-8a4b-71f9a3c33aed None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:23.736259 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dffbb69-2662-4b04-8ee9-802cb03e51fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:23.736922 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dffbb69-2662-4b04-8ee9-802cb03e51fd None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:23.737206 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dffbb69-2662-4b04-8ee9-802cb03e51fd None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:23.737480 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dffbb69-2662-4b04-8ee9-802cb03e51fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:23.738222 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dffbb69-2662-4b04-8ee9-802cb03e51fd None admin] RBAC: Adding query filter params (role.id=163c30c3b3c246beb849ce176f02d9da, scope.project.id=e07dc5ebdc504c6da706a2d21c0a3970, user.id=e7f43e2756b746a6825ca1a06b7d7c91) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:23.738528 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dffbb69-2662-4b04-8ee9-802cb03e51fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:23.740271 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dffbb69-2662-4b04-8ee9-802cb03e51fd None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:23.744975 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b81b83-0828-4388-b79a-9c3d908d40de None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:23.746771 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 411/834] 10.222.0.61 () {56 vars in 1374 bytes} [Mon Aug 22 19:06:23 2022] GET /identity/v3/role_assignments?user.id=e7f43e2756b746a6825ca1a06b7d7c91&scope.project.id=e07dc5ebdc504c6da706a2d21c0a3970&role.id=163c30c3b3c246beb849ce176f02d9da => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:23.817685 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:23.864967 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b81b83-0828-4388-b79a-9c3d908d40de None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:23.901861 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3f0f53-c7c1-47c4-aebc-2ee1dc244a38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:23.902507 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3f0f53-c7c1-47c4-aebc-2ee1dc244a38 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:23.902843 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3f0f53-c7c1-47c4-aebc-2ee1dc244a38 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:23.903061 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3f0f53-c7c1-47c4-aebc-2ee1dc244a38 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:23.913834 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3f0f53-c7c1-47c4-aebc-2ee1dc244a38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 19:06:23.920346 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-cf3f0f53-c7c1-47c4-aebc-2ee1dc244a38 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 19:06:23.920346 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:23.920346 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:23.920346 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:23.920346 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:23.920346 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:23.920346 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:23.920346 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:23.920346 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:23.920346 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:23.920346 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:23.920346 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:23.920346 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 19:06:23.920346 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 19:06:23.920346 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 19:06:23.920346 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 19:06:23.921825 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 19:06:23.921825 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 19:06:23.921825 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 19:06:23.921825 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 19:06:23.921825 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:23.921825 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:23.921825 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:23.921825 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:23.921825 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:23.921825 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:23.921825 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:23.921825 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:23.921825 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:23.921825 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:23.921825 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:23.921825 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:23.923173 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:23.923173 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:23.923173 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:23.923173 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:23.923173 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 19:06:23.923173 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 19:06:23.923173 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 19:06:23.923173 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 19:06:23.923173 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 19:06:23.923173 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 19:06:23.923173 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 19:06:23.923173 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:06:23.923173 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 424/835] 10.222.0.61 () {56 vars in 1096 bytes} [Mon Aug 22 19:06:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 19:06:23.928179 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dffbb69-2662-4b04-8ee9-802cb03e51fd None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:24.007398 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:24.039122 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dffbb69-2662-4b04-8ee9-802cb03e51fd None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:24.075118 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edf9e40-6af8-4c44-9045-9da1cbb554a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:24.075793 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edf9e40-6af8-4c44-9045-9da1cbb554a1 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:24.076080 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edf9e40-6af8-4c44-9045-9da1cbb554a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:24.076356 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edf9e40-6af8-4c44-9045-9da1cbb554a1 None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:24.077062 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edf9e40-6af8-4c44-9045-9da1cbb554a1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:24.077404 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edf9e40-6af8-4c44-9045-9da1cbb554a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:24.078818 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edf9e40-6af8-4c44-9045-9da1cbb554a1 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:24.088470 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 412/836] 10.222.0.61 () {56 vars in 1099 bytes} [Mon Aug 22 19:06:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:24.094132 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3f0f53-c7c1-47c4-aebc-2ee1dc244a38 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:24.205467 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3f0f53-c7c1-47c4-aebc-2ee1dc244a38 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:24.225809 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:24.241341 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6579419-0bd1-48ff-aac9-ee5bbe0cfde1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:24.241972 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6579419-0bd1-48ff-aac9-ee5bbe0cfde1 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:24.242246 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6579419-0bd1-48ff-aac9-ee5bbe0cfde1 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:24.242515 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6579419-0bd1-48ff-aac9-ee5bbe0cfde1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:24.253420 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6579419-0bd1-48ff-aac9-ee5bbe0cfde1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:24.254873 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6579419-0bd1-48ff-aac9-ee5bbe0cfde1 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:24.268876 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-c6579419-0bd1-48ff-aac9-ee5bbe0cfde1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:06:24.268876 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:24.268876 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:24.268876 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:24.268876 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:24.268876 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:24.268876 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:24.268876 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:24.268876 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:24.268876 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:24.268876 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:24.268876 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:24.268876 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 19:06:24.268876 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 19:06:24.268876 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 19:06:24.268876 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 19:06:24.269999 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:24.269999 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:24.269999 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:24.269999 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:24.269999 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:24.269999 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:24.269999 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:24.269999 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:24.269999 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:24.269999 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:24.269999 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:24.269999 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:24.269999 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:24.269999 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:24.269999 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:24.269999 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:24.269999 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 19:06:24.271094 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 19:06:24.271094 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:06:24.271094 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:06:24.271094 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 425/837] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:06:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:24.277277 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edf9e40-6af8-4c44-9045-9da1cbb554a1 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:24.389264 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edf9e40-6af8-4c44-9045-9da1cbb554a1 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:24.428943 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c089dd-4858-46e6-98d5-a8eb62d93139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:24.429714 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c089dd-4858-46e6-98d5-a8eb62d93139 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:24.430096 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c089dd-4858-46e6-98d5-a8eb62d93139 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:24.430477 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c089dd-4858-46e6-98d5-a8eb62d93139 None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:24.431234 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c089dd-4858-46e6-98d5-a8eb62d93139 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:24.431654 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c089dd-4858-46e6-98d5-a8eb62d93139 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:24.433126 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c089dd-4858-46e6-98d5-a8eb62d93139 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:24.434783 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:24.446601 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 413/838] 10.222.0.61 () {56 vars in 1109 bytes} [Mon Aug 22 19:06:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:24.452117 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6579419-0bd1-48ff-aac9-ee5bbe0cfde1 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:24.557153 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6579419-0bd1-48ff-aac9-ee5bbe0cfde1 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:24.597205 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706c0373-8158-4b79-b0ef-df53421a07de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:24.597860 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706c0373-8158-4b79-b0ef-df53421a07de None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:24.598137 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706c0373-8158-4b79-b0ef-df53421a07de None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:24.598405 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706c0373-8158-4b79-b0ef-df53421a07de None admin] PATH_INFO: `/v3/users/swift` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:24.606799 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706c0373-8158-4b79-b0ef-df53421a07de None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:24.608275 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706c0373-8158-4b79-b0ef-df53421a07de None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:24.620021 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-706c0373-8158-4b79-b0ef-df53421a07de None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 22 19:06:24.620021 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:24.620021 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:24.620021 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:24.620021 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:24.620021 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:24.620021 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:24.620021 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:24.620021 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:24.620021 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:24.620021 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:24.620021 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:24.620021 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 19:06:24.620021 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 19:06:24.620021 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 19:06:24.620021 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 19:06:24.621184 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:24.621184 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:24.621184 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 19:06:24.621184 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 19:06:24.621184 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 19:06:24.621184 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 19:06:24.621184 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:24.621184 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:24.621184 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:24.621184 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:24.621184 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:24.621184 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:24.621184 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:24.621184 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:24.621184 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:24.621184 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:24.621184 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:24.621184 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:24.622393 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:24.622393 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:24.622393 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 19:06:24.622393 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 19:06:24.622393 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 19:06:24.622393 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 19:06:24.622393 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 19:06:24.622393 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 19:06:24.622393 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 22 19:06:24.622393 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:06:24.622393 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 426/839] 10.222.0.61 () {56 vars in 1096 bytes} [Mon Aug 22 19:06:24 2022] GET /identity/v3/users/swift => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:24.629137 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c089dd-4858-46e6-98d5-a8eb62d93139 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:24.730702 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c089dd-4858-46e6-98d5-a8eb62d93139 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:24.766482 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122efdcb-9bf9-4dd9-a17c-afafeebd2686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:24.767365 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122efdcb-9bf9-4dd9-a17c-afafeebd2686 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:24.767834 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122efdcb-9bf9-4dd9-a17c-afafeebd2686 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:24.768272 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122efdcb-9bf9-4dd9-a17c-afafeebd2686 None admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:24.769168 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122efdcb-9bf9-4dd9-a17c-afafeebd2686 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:24.769621 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122efdcb-9bf9-4dd9-a17c-afafeebd2686 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:24.771185 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122efdcb-9bf9-4dd9-a17c-afafeebd2686 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:24.789555 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 414/840] 10.222.0.61 () {56 vars in 1135 bytes} [Mon Aug 22 19:06:24 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:24.795383 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706c0373-8158-4b79-b0ef-df53421a07de None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:24.828386 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:24.900462 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706c0373-8158-4b79-b0ef-df53421a07de None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:24.935193 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2136787-0c28-40bd-8d50-76a1cb9d5f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:24.935847 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2136787-0c28-40bd-8d50-76a1cb9d5f6d None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:24.936129 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2136787-0c28-40bd-8d50-76a1cb9d5f6d None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:24.936402 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2136787-0c28-40bd-8d50-76a1cb9d5f6d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:24.949021 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d240f7-f8c8-4001-88a9-73070177a875 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:24.949606 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d240f7-f8c8-4001-88a9-73070177a875 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:24.950080 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d240f7-f8c8-4001-88a9-73070177a875 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:24.951912 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 415/841] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:06:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 19:06:24.957073 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305c9092-ddd7-40e7-8c0c-75ec243e8317 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:24.957471 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305c9092-ddd7-40e7-8c0c-75ec243e8317 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:24.957847 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305c9092-ddd7-40e7-8c0c-75ec243e8317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:24.978460 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-305c9092-ddd7-40e7-8c0c-75ec243e8317 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:06:24.979824 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-305c9092-ddd7-40e7-8c0c-75ec243e8317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:06:25.017092 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2136787-0c28-40bd-8d50-76a1cb9d5f6d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:25.018963 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2136787-0c28-40bd-8d50-76a1cb9d5f6d None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:25.044532 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:25.054663 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:25.075306 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 416/842] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:06:24 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 22 19:06:25.143393 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 427/843] 10.222.0.61 () {58 vars in 1282 bytes} [Mon Aug 22 19:06:24 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 22 19:06:25.149351 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305c9092-ddd7-40e7-8c0c-75ec243e8317 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:25.253900 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305c9092-ddd7-40e7-8c0c-75ec243e8317 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:25.289058 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba476bfa-9fd3-496d-a97b-d00b827978cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:25.289698 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba476bfa-9fd3-496d-a97b-d00b827978cf None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:25.289978 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba476bfa-9fd3-496d-a97b-d00b827978cf None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:25.290250 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba476bfa-9fd3-496d-a97b-d00b827978cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:25.300921 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba476bfa-9fd3-496d-a97b-d00b827978cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:25.302358 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba476bfa-9fd3-496d-a97b-d00b827978cf None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:25.316904 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-ba476bfa-9fd3-496d-a97b-d00b827978cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:06:25.316904 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:25.316904 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:25.316904 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:25.316904 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:25.316904 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:25.316904 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:25.316904 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:25.316904 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:25.316904 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:25.316904 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:25.316904 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:25.316904 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 19:06:25.316904 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 19:06:25.316904 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 19:06:25.316904 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 19:06:25.317873 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:25.317873 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:25.317873 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:25.317873 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:25.317873 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:25.317873 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:25.317873 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:25.317873 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:25.317873 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:25.317873 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:25.317873 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:25.317873 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:25.317873 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:25.317873 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:25.317873 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:25.317873 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:25.317873 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 19:06:25.318759 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 19:06:25.318759 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:06:25.318759 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:06:25.318759 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 417/844] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:06:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:25.323891 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2136787-0c28-40bd-8d50-76a1cb9d5f6d None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:25.357813 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba476bfa-9fd3-496d-a97b-d00b827978cf None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:25.394287 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:25.449876 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2136787-0c28-40bd-8d50-76a1cb9d5f6d None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:25.466381 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba476bfa-9fd3-496d-a97b-d00b827978cf None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:25.505056 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baa1f6d-fe80-4120-aeda-ad7e4f3a9f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:25.505802 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baa1f6d-fe80-4120-aeda-ad7e4f3a9f10 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:25.506081 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baa1f6d-fe80-4120-aeda-ad7e4f3a9f10 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:25.506389 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baa1f6d-fe80-4120-aeda-ad7e4f3a9f10 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:25.519960 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baa1f6d-fe80-4120-aeda-ad7e4f3a9f10 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 19:06:25.525376 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-1baa1f6d-fe80-4120-aeda-ad7e4f3a9f10 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 19:06:25.525376 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:25.525376 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:25.525376 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:25.525376 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:25.525376 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:25.525376 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:25.525376 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:25.525376 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:25.525376 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:25.525376 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:25.525376 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:25.525376 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 19:06:25.525376 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 19:06:25.525376 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 19:06:25.525376 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 19:06:25.526377 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 19:06:25.526377 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 19:06:25.526377 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 19:06:25.526377 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 19:06:25.526377 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:25.526377 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:25.526377 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:25.526377 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:25.526377 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:25.526377 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:25.526377 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:25.526377 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:25.526377 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:25.526377 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:25.526377 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:25.526377 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:25.527217 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:25.527217 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:25.527217 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:25.527217 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:25.527217 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 19:06:25.527217 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 19:06:25.527217 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 19:06:25.527217 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 19:06:25.527217 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 19:06:25.527217 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 19:06:25.527217 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 19:06:25.527217 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:06:25.532405 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 418/845] 10.222.0.61 () {56 vars in 1102 bytes} [Mon Aug 22 19:06:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 19:06:25.533752 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450e9ef5-0cd9-45a9-a9cc-446406bb38cc None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:25.534033 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450e9ef5-0cd9-45a9-a9cc-446406bb38cc None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:25.534304 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450e9ef5-0cd9-45a9-a9cc-446406bb38cc None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:25.539702 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 419/846] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:06:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:06:25.539702 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450e9ef5-0cd9-45a9-a9cc-446406bb38cc None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:25.605806 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:25.612365 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52b4254-cad1-4e1a-a42f-b3456e0c449c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:25.612976 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52b4254-cad1-4e1a-a42f-b3456e0c449c None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:25.613254 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52b4254-cad1-4e1a-a42f-b3456e0c449c None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:25.613524 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52b4254-cad1-4e1a-a42f-b3456e0c449c None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:25.614432 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52b4254-cad1-4e1a-a42f-b3456e0c449c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:25.614739 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52b4254-cad1-4e1a-a42f-b3456e0c449c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:25.616139 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52b4254-cad1-4e1a-a42f-b3456e0c449c None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:25.628637 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 428/847] 10.222.0.61 () {56 vars in 1109 bytes} [Mon Aug 22 19:06:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:25.629868 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e56f6b-bb6a-47ad-9f88-a859d1279f8b None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:25.630158 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e56f6b-bb6a-47ad-9f88-a859d1279f8b None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:25.630436 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e56f6b-bb6a-47ad-9f88-a859d1279f8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:25.648089 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-a7e56f6b-bb6a-47ad-9f88-a859d1279f8b None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:06:25.649220 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-a7e56f6b-bb6a-47ad-9f88-a859d1279f8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:06:25.650034 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:25.660253 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450e9ef5-0cd9-45a9-a9cc-446406bb38cc None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:25.696161 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e75a508-87d4-4cde-9f47-9a560eb5be32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:25.697007 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e75a508-87d4-4cde-9f47-9a560eb5be32 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:25.697133 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e75a508-87d4-4cde-9f47-9a560eb5be32 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:25.697313 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e75a508-87d4-4cde-9f47-9a560eb5be32 None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:25.697970 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e75a508-87d4-4cde-9f47-9a560eb5be32 None admin] RBAC: Adding query filter params (name=service) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:25.698359 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e75a508-87d4-4cde-9f47-9a560eb5be32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:25.699636 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e75a508-87d4-4cde-9f47-9a560eb5be32 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:25.710607 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 420/848] 10.222.0.61 () {56 vars in 1103 bytes} [Mon Aug 22 19:06:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:25.711213 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e75a508-87d4-4cde-9f47-9a560eb5be32 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:25.720648 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 429/849] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:06:25 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 19:06:25.721500 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e56f6b-bb6a-47ad-9f88-a859d1279f8b None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:25.827629 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e56f6b-bb6a-47ad-9f88-a859d1279f8b None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:25.837141 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e75a508-87d4-4cde-9f47-9a560eb5be32 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:25.851594 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:25.865362 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f0f005-4b51-4603-b574-524ed8db074f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:25.866113 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f0f005-4b51-4603-b574-524ed8db074f None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:25.866490 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f0f005-4b51-4603-b574-524ed8db074f None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:25.866828 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f0f005-4b51-4603-b574-524ed8db074f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:25.874334 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451db749-5488-4e00-98ef-9a01ce4f4d0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:25.875226 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451db749-5488-4e00-98ef-9a01ce4f4d0a None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:25.875732 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451db749-5488-4e00-98ef-9a01ce4f4d0a None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:25.876197 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451db749-5488-4e00-98ef-9a01ce4f4d0a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:25.876776 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f0f005-4b51-4603-b574-524ed8db074f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:25.878209 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f0f005-4b51-4603-b574-524ed8db074f None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:25.883548 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451db749-5488-4e00-98ef-9a01ce4f4d0a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:25.885273 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451db749-5488-4e00-98ef-9a01ce4f4d0a None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:25.892344 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-52f0f005-4b51-4603-b574-524ed8db074f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:06:25.892344 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:25.892344 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:25.892344 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:25.892344 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:25.892344 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:25.892344 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:25.892344 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:25.892344 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:25.892344 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:25.892344 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:25.892344 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:25.892344 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 19:06:25.892344 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 19:06:25.892344 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 19:06:25.892344 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 19:06:25.893301 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:25.893301 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:25.893301 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:25.893301 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:25.893301 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:25.893301 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:25.893301 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:25.893301 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:25.893301 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:25.893301 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:25.893301 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:25.893301 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:25.893301 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:25.893301 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:25.893301 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:25.893301 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:25.893301 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 19:06:25.894224 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 19:06:25.894224 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:06:25.894224 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:06:25.894224 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 430/850] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:06:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:25.894744 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb687b0-8bf9-42ce-8014-30ce15a59708 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:25.894915 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb687b0-8bf9-42ce-8014-30ce15a59708 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:25.895142 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb687b0-8bf9-42ce-8014-30ce15a59708 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:25.896657 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 431/851] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:06:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:06:25.898898 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb687b0-8bf9-42ce-8014-30ce15a59708 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:25.900460 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-451db749-5488-4e00-98ef-9a01ce4f4d0a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 19:06:25.900460 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:25.900460 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:25.900460 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:25.900460 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:25.900460 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:25.900460 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:25.900460 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:25.900460 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:25.900460 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:25.900460 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:25.900460 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:25.900460 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 19:06:25.900460 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 19:06:25.900460 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 19:06:25.901531 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 19:06:25.901531 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:25.901531 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:25.901531 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:25.901531 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:25.901531 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:25.901531 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:25.901531 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:25.901531 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:25.901531 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:25.901531 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:25.901531 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:25.901531 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:25.901531 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:25.901531 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:25.901531 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:25.901531 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:25.902560 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 19:06:25.902560 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 19:06:25.902560 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 19:06:25.902560 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 19:06:25.902560 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 19:06:25.902560 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 19:06:25.902560 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 19:06:25.902560 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:06:25.910284 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 421/852] 10.222.0.61 () {56 vars in 1111 bytes} [Mon Aug 22 19:06:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:25.910284 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf43b7f8-f0de-46f7-9b5e-df8b1411f602 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:25.910284 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf43b7f8-f0de-46f7-9b5e-df8b1411f602 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:25.910284 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf43b7f8-f0de-46f7-9b5e-df8b1411f602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:25.924519 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-cf43b7f8-f0de-46f7-9b5e-df8b1411f602 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:06:25.925691 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-cf43b7f8-f0de-46f7-9b5e-df8b1411f602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:06:25.937624 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:25.994893 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 422/853] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:06:25 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 19:06:25.995768 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf43b7f8-f0de-46f7-9b5e-df8b1411f602 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:26.018268 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb687b0-8bf9-42ce-8014-30ce15a59708 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:26.054576 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b33e8d-4eab-409d-9f8f-edfcdd9a6673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:26.055175 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b33e8d-4eab-409d-9f8f-edfcdd9a6673 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:26.055449 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b33e8d-4eab-409d-9f8f-edfcdd9a6673 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:26.055749 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b33e8d-4eab-409d-9f8f-edfcdd9a6673 None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:26.056378 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b33e8d-4eab-409d-9f8f-edfcdd9a6673 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:26.056673 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b33e8d-4eab-409d-9f8f-edfcdd9a6673 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:26.058027 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b33e8d-4eab-409d-9f8f-edfcdd9a6673 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:26.069977 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 432/854] 10.222.0.61 () {56 vars in 1109 bytes} [Mon Aug 22 19:06:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:26.070866 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b33e8d-4eab-409d-9f8f-edfcdd9a6673 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:26.106036 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:26.116627 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf43b7f8-f0de-46f7-9b5e-df8b1411f602 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:26.160219 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be9ba77-528d-4766-95ca-4e609743faf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:26.160819 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be9ba77-528d-4766-95ca-4e609743faf9 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:26.161047 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be9ba77-528d-4766-95ca-4e609743faf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:26.161298 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be9ba77-528d-4766-95ca-4e609743faf9 None admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:26.161993 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be9ba77-528d-4766-95ca-4e609743faf9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:26.162709 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be9ba77-528d-4766-95ca-4e609743faf9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:26.164408 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be9ba77-528d-4766-95ca-4e609743faf9 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:26.174248 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b33e8d-4eab-409d-9f8f-edfcdd9a6673 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:26.183506 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 423/855] 10.222.0.61 () {56 vars in 1148 bytes} [Mon Aug 22 19:06:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:26.183506 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be9ba77-528d-4766-95ca-4e609743faf9 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:26.193509 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:26.212927 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77043c61-a897-4b8a-a4e7-0b95a1f8702a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:26.213557 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77043c61-a897-4b8a-a4e7-0b95a1f8702a None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:26.213840 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77043c61-a897-4b8a-a4e7-0b95a1f8702a None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:26.214108 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77043c61-a897-4b8a-a4e7-0b95a1f8702a None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:26.214683 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77043c61-a897-4b8a-a4e7-0b95a1f8702a None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:26.216275 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77043c61-a897-4b8a-a4e7-0b95a1f8702a None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:26.225534 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-77043c61-a897-4b8a-a4e7-0b95a1f8702a None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 22 19:06:26.225534 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:26.225534 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 19:06:26.225534 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 19:06:26.225534 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 19:06:26.225534 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 19:06:26.225534 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 19:06:26.225534 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 19:06:26.225534 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 22 19:06:26.225534 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:06:26.225534 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 19:06:26.225534 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:06:26.225534 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:26.225534 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:26.225534 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:26.225534 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:26.225534 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:26.226552 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:26.226552 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:26.226552 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:26.226552 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:26.226552 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:26.226552 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:26.226552 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 19:06:26.226552 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 19:06:26.226552 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 19:06:26.226552 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 19:06:26.226552 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:26.226552 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:26.226552 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:26.226552 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:26.226552 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:26.226552 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:26.227408 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:26.227408 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:26.227408 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:26.227408 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:26.227408 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:26.227408 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:26.227408 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:26.227408 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:26.227408 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:26.227408 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:26.227408 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 19:06:26.227408 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 19:06:26.227408 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 22 19:06:26.227408 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:06:26.227408 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 433/856] 10.222.0.61 () {56 vars in 1114 bytes} [Mon Aug 22 19:06:26 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:26.228261 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77043c61-a897-4b8a-a4e7-0b95a1f8702a None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:26.291857 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be9ba77-528d-4766-95ca-4e609743faf9 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:26.323166 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:26.326362 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68e23cb-5f80-48ec-b93e-1797d284e9b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:26.327062 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68e23cb-5f80-48ec-b93e-1797d284e9b7 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:26.327476 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77043c61-a897-4b8a-a4e7-0b95a1f8702a None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:26.327570 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68e23cb-5f80-48ec-b93e-1797d284e9b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:26.327668 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68e23cb-5f80-48ec-b93e-1797d284e9b7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:26.335113 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68e23cb-5f80-48ec-b93e-1797d284e9b7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:26.336566 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68e23cb-5f80-48ec-b93e-1797d284e9b7 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:26.347575 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-d68e23cb-5f80-48ec-b93e-1797d284e9b7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 22 19:06:26.347575 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:26.347575 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:26.347575 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:26.347575 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:26.347575 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:26.347575 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:26.347575 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:26.347575 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:26.347575 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:26.347575 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:26.347575 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:26.347575 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 19:06:26.347575 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 19:06:26.347575 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 19:06:26.347575 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 19:06:26.348701 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:26.348701 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:26.348701 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 19:06:26.348701 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 19:06:26.348701 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 19:06:26.348701 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 19:06:26.348701 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:26.348701 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:26.348701 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:26.348701 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:26.348701 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:26.348701 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:26.348701 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:26.348701 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:26.348701 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:26.348701 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:26.348701 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:26.348701 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:26.349851 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:26.349851 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:26.349851 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 19:06:26.349851 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 19:06:26.349851 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 19:06:26.349851 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 19:06:26.349851 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 19:06:26.349851 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 19:06:26.349851 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 22 19:06:26.349851 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:06:26.349851 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 424/857] 10.222.0.61 () {56 vars in 1117 bytes} [Mon Aug 22 19:06:26 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:26.349851 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68e23cb-5f80-48ec-b93e-1797d284e9b7 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:26.363535 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1cc1d1-b5ee-4ec8-abd1-bdfa5e2e7fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:26.364415 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1cc1d1-b5ee-4ec8-abd1-bdfa5e2e7fb0 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:26.364698 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1cc1d1-b5ee-4ec8-abd1-bdfa5e2e7fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:26.364959 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1cc1d1-b5ee-4ec8-abd1-bdfa5e2e7fb0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:26.365678 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1cc1d1-b5ee-4ec8-abd1-bdfa5e2e7fb0 None admin] RBAC: Adding query filter params (role.id=163c30c3b3c246beb849ce176f02d9da, scope.project.id=e07dc5ebdc504c6da706a2d21c0a3970, user.id=2824359491934d0dacaa21a718becee5) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:26.365982 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1cc1d1-b5ee-4ec8-abd1-bdfa5e2e7fb0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:26.367385 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1cc1d1-b5ee-4ec8-abd1-bdfa5e2e7fb0 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:26.373022 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 434/858] 10.222.0.61 () {56 vars in 1374 bytes} [Mon Aug 22 19:06:26 2022] GET /identity/v3/role_assignments?user.id=2824359491934d0dacaa21a718becee5&scope.project.id=e07dc5ebdc504c6da706a2d21c0a3970&role.id=163c30c3b3c246beb849ce176f02d9da => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:26.373784 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1cc1d1-b5ee-4ec8-abd1-bdfa5e2e7fb0 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:26.423984 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:26.466618 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68e23cb-5f80-48ec-b93e-1797d284e9b7 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:26.495986 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1cc1d1-b5ee-4ec8-abd1-bdfa5e2e7fb0 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:26.504912 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065085d9-56a3-43d7-9b06-647a932c2f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:26.505529 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065085d9-56a3-43d7-9b06-647a932c2f34 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:26.505805 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065085d9-56a3-43d7-9b06-647a932c2f34 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:26.506073 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065085d9-56a3-43d7-9b06-647a932c2f34 None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:26.506739 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065085d9-56a3-43d7-9b06-647a932c2f34 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:26.507065 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065085d9-56a3-43d7-9b06-647a932c2f34 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:26.509432 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065085d9-56a3-43d7-9b06-647a932c2f34 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:26.515577 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 425/859] 10.222.0.61 () {56 vars in 1114 bytes} [Mon Aug 22 19:06:26 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:26.516342 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065085d9-56a3-43d7-9b06-647a932c2f34 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:26.540267 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ee9830-7e70-463a-a5f7-e146fc700d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:26.540900 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ee9830-7e70-463a-a5f7-e146fc700d4c None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:26.541177 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ee9830-7e70-463a-a5f7-e146fc700d4c None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:26.541444 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ee9830-7e70-463a-a5f7-e146fc700d4c None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:26.542388 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ee9830-7e70-463a-a5f7-e146fc700d4c None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:26.544129 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ee9830-7e70-463a-a5f7-e146fc700d4c None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:26.557744 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-f8ee9830-7e70-463a-a5f7-e146fc700d4c None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 22 19:06:26.557744 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:26.557744 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 19:06:26.557744 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 19:06:26.557744 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 19:06:26.557744 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 19:06:26.557744 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 19:06:26.557744 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 19:06:26.557744 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 22 19:06:26.557744 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:06:26.557744 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 19:06:26.557744 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:06:26.557744 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:26.557744 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:26.557744 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:26.557744 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:26.558790 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:26.558790 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:26.558790 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:26.558790 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:26.558790 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:26.558790 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:26.558790 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:26.558790 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 19:06:26.558790 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 19:06:26.558790 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 19:06:26.558790 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 19:06:26.558790 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:26.558790 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:26.558790 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:26.558790 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:26.558790 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:26.558790 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:26.559848 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:26.559848 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:26.559848 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:26.559848 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:26.559848 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:26.559848 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:26.559848 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:26.559848 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:26.559848 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:26.559848 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:26.559848 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 19:06:26.559848 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 19:06:26.559848 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 22 19:06:26.559848 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:06:26.563716 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 435/860] 10.222.0.61 () {56 vars in 1132 bytes} [Mon Aug 22 19:06:26 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:26.564570 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ee9830-7e70-463a-a5f7-e146fc700d4c None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:26.595020 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:26.632606 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065085d9-56a3-43d7-9b06-647a932c2f34 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:26.662199 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:26.670305 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94fe588-4cfb-4532-ba32-879928efc510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:26.671492 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ee9830-7e70-463a-a5f7-e146fc700d4c None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:26.671870 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94fe588-4cfb-4532-ba32-879928efc510 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:26.672302 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94fe588-4cfb-4532-ba32-879928efc510 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:26.672722 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94fe588-4cfb-4532-ba32-879928efc510 None admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:26.674798 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94fe588-4cfb-4532-ba32-879928efc510 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:26.675270 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94fe588-4cfb-4532-ba32-879928efc510 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:26.676846 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94fe588-4cfb-4532-ba32-879928efc510 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:26.700153 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 426/861] 10.222.0.61 () {56 vars in 1149 bytes} [Mon Aug 22 19:06:26 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:26.701698 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94fe588-4cfb-4532-ba32-879928efc510 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:26.708373 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19dea48-f598-4e2d-a91f-2990bb7cc904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:26.708373 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19dea48-f598-4e2d-a91f-2990bb7cc904 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:26.708636 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19dea48-f598-4e2d-a91f-2990bb7cc904 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:26.708862 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19dea48-f598-4e2d-a91f-2990bb7cc904 None admin] PATH_INFO: `/v3/services` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:26.709488 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19dea48-f598-4e2d-a91f-2990bb7cc904 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:26.709786 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19dea48-f598-4e2d-a91f-2990bb7cc904 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:26.711151 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19dea48-f598-4e2d-a91f-2990bb7cc904 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:26.716140 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 436/862] 10.222.0.61 () {56 vars in 1114 bytes} [Mon Aug 22 19:06:26 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:26.716923 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19dea48-f598-4e2d-a91f-2990bb7cc904 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:26.834854 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94fe588-4cfb-4532-ba32-879928efc510 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:26.860630 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:26.867123 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19dea48-f598-4e2d-a91f-2990bb7cc904 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:26.880444 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddc0546-a534-4752-8e14-bc96278977aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:26.881183 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddc0546-a534-4752-8e14-bc96278977aa None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:26.881464 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddc0546-a534-4752-8e14-bc96278977aa None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:26.881736 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddc0546-a534-4752-8e14-bc96278977aa None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:26.882570 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddc0546-a534-4752-8e14-bc96278977aa None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:26.884718 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddc0546-a534-4752-8e14-bc96278977aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:26.884835 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddc0546-a534-4752-8e14-bc96278977aa None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:26.895719 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 427/863] 10.222.0.61 () {56 vars in 1126 bytes} [Mon Aug 22 19:06:26 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:26.903909 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddc0546-a534-4752-8e14-bc96278977aa None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:26.916402 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeeb812-eefd-441d-a31a-61943da0d001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:26.917060 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeeb812-eefd-441d-a31a-61943da0d001 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:26.917575 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeeb812-eefd-441d-a31a-61943da0d001 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:26.917575 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeeb812-eefd-441d-a31a-61943da0d001 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:26.960018 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:26.990605 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aeeb812-eefd-441d-a31a-61943da0d001 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:26.992066 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aeeb812-eefd-441d-a31a-61943da0d001 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:27.009056 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddc0546-a534-4752-8e14-bc96278977aa None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:27.046483 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3d279d-af72-4ae0-9a51-488e9c6bd39c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:27.047459 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3d279d-af72-4ae0-9a51-488e9c6bd39c None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:27.048003 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3d279d-af72-4ae0-9a51-488e9c6bd39c None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:27.048505 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3d279d-af72-4ae0-9a51-488e9c6bd39c None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:27.049351 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3d279d-af72-4ae0-9a51-488e9c6bd39c None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:27.049853 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3d279d-af72-4ae0-9a51-488e9c6bd39c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:27.051417 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3d279d-af72-4ae0-9a51-488e9c6bd39c None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:27.056791 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 428/864] 10.222.0.61 () {56 vars in 1126 bytes} [Mon Aug 22 19:06:26 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:27.102452 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:27.109599 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 437/865] 10.222.0.61 () {58 vars in 1282 bytes} [Mon Aug 22 19:06:26 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 22 19:06:27.112495 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3d279d-af72-4ae0-9a51-488e9c6bd39c None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:27.227289 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3d279d-af72-4ae0-9a51-488e9c6bd39c None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:27.244597 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:27.264083 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038ed6cd-f31f-4e7f-941b-10b30d81524b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:27.265651 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038ed6cd-f31f-4e7f-941b-10b30d81524b None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:27.266363 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038ed6cd-f31f-4e7f-941b-10b30d81524b None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:27.266699 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038ed6cd-f31f-4e7f-941b-10b30d81524b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:27.277365 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038ed6cd-f31f-4e7f-941b-10b30d81524b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:27.278962 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038ed6cd-f31f-4e7f-941b-10b30d81524b None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:27.293997 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-038ed6cd-f31f-4e7f-941b-10b30d81524b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:06:27.293997 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:27.293997 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:27.293997 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:27.293997 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:27.293997 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:27.293997 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:27.293997 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:27.293997 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:27.293997 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:27.293997 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:27.293997 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:27.293997 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 19:06:27.293997 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 19:06:27.293997 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 19:06:27.293997 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 19:06:27.295118 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:27.295118 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:27.295118 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:27.295118 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:27.295118 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:27.295118 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:27.295118 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:27.295118 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:27.295118 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:27.295118 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:27.295118 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:27.295118 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:27.295118 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:27.295118 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:27.295118 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:27.295118 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:27.295118 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 19:06:27.296126 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 19:06:27.296126 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:06:27.296126 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:06:27.299968 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 429/866] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:06:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:27.301477 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeeb812-eefd-441d-a31a-61943da0d001 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:27.409195 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeeb812-eefd-441d-a31a-61943da0d001 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:27.444469 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969367c8-1892-4841-a6df-eae6c0bb6379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:27.445075 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969367c8-1892-4841-a6df-eae6c0bb6379 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:27.445350 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969367c8-1892-4841-a6df-eae6c0bb6379 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:27.445621 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969367c8-1892-4841-a6df-eae6c0bb6379 None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:27.446268 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969367c8-1892-4841-a6df-eae6c0bb6379 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:27.446569 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969367c8-1892-4841-a6df-eae6c0bb6379 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:27.448030 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969367c8-1892-4841-a6df-eae6c0bb6379 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:27.460558 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 438/867] 10.222.0.61 () {56 vars in 1109 bytes} [Mon Aug 22 19:06:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:27.465967 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038ed6cd-f31f-4e7f-941b-10b30d81524b None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:27.574395 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038ed6cd-f31f-4e7f-941b-10b30d81524b None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:27.609677 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824979f3-8466-4902-807e-03e2d50b85db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:27.610317 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824979f3-8466-4902-807e-03e2d50b85db None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:27.610594 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824979f3-8466-4902-807e-03e2d50b85db None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:27.610912 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824979f3-8466-4902-807e-03e2d50b85db None admin] PATH_INFO: `/v3/projects/service` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:27.618032 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824979f3-8466-4902-807e-03e2d50b85db None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:27.619531 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824979f3-8466-4902-807e-03e2d50b85db None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:27.631353 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-824979f3-8466-4902-807e-03e2d50b85db None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 19:06:27.631353 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:27.631353 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:27.631353 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:27.631353 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:27.631353 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:27.631353 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:27.631353 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:27.631353 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:27.631353 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:27.631353 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:27.631353 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:27.631353 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 19:06:27.631353 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 19:06:27.631353 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 19:06:27.632285 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 19:06:27.632285 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:27.632285 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:27.632285 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:27.632285 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:27.632285 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:27.632285 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:27.632285 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:27.632285 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:27.632285 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:27.632285 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:27.632285 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:27.632285 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:27.632285 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:27.632285 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:27.632285 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:27.632285 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:27.633582 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 19:06:27.633582 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 19:06:27.633582 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 19:06:27.633582 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 19:06:27.633582 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 19:06:27.633582 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 19:06:27.633582 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 19:06:27.633582 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:06:27.633582 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 430/868] 10.222.0.61 () {56 vars in 1111 bytes} [Mon Aug 22 19:06:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:27.638402 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969367c8-1892-4841-a6df-eae6c0bb6379 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:27.691432 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:27.745020 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969367c8-1892-4841-a6df-eae6c0bb6379 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:27.780849 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bc651f-b027-4827-8458-7d312d401108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:27.781825 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bc651f-b027-4827-8458-7d312d401108 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:27.782313 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bc651f-b027-4827-8458-7d312d401108 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:27.782818 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bc651f-b027-4827-8458-7d312d401108 None admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:27.783753 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bc651f-b027-4827-8458-7d312d401108 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:27.784268 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bc651f-b027-4827-8458-7d312d401108 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:27.785961 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bc651f-b027-4827-8458-7d312d401108 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:27.799739 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 439/869] 10.222.0.61 () {56 vars in 1148 bytes} [Mon Aug 22 19:06:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:27.807824 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824979f3-8466-4902-807e-03e2d50b85db None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:27.837067 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:27.911793 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824979f3-8466-4902-807e-03e2d50b85db None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:27.947152 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aaedfd-3a0c-427d-ae94-912d1e80e160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:27.947778 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aaedfd-3a0c-427d-ae94-912d1e80e160 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:27.948060 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aaedfd-3a0c-427d-ae94-912d1e80e160 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:27.948334 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aaedfd-3a0c-427d-ae94-912d1e80e160 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:27.949063 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11aaedfd-3a0c-427d-ae94-912d1e80e160 None admin] RBAC: Adding query filter params (role.id=324bd04d55c34b61a5282c9f6ed73f46, scope.project.id=e07dc5ebdc504c6da706a2d21c0a3970, user.id=399396a8fe254da199bc48b0ca5243d0) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:27.949369 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11aaedfd-3a0c-427d-ae94-912d1e80e160 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:27.950791 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11aaedfd-3a0c-427d-ae94-912d1e80e160 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:27.957192 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 431/870] 10.222.0.61 () {56 vars in 1374 bytes} [Mon Aug 22 19:06:27 2022] GET /identity/v3/role_assignments?user.id=399396a8fe254da199bc48b0ca5243d0&scope.project.id=e07dc5ebdc504c6da706a2d21c0a3970&role.id=324bd04d55c34b61a5282c9f6ed73f46 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:28.413120 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05265aab-5039-40d1-b553-523079bd0457 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:28.413641 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05265aab-5039-40d1-b553-523079bd0457 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:28.413641 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05265aab-5039-40d1-b553-523079bd0457 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:28.415212 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 440/871] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:06:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:06:28.420540 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bae62e5-6b4e-453e-bc56-dd0e871cae3a None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:28.420985 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bae62e5-6b4e-453e-bc56-dd0e871cae3a None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:28.421159 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bae62e5-6b4e-453e-bc56-dd0e871cae3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:28.439540 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-8bae62e5-6b4e-453e-bc56-dd0e871cae3a None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:06:28.440789 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-8bae62e5-6b4e-453e-bc56-dd0e871cae3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:06:28.511050 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 432/872] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:06:28 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 19:06:28.686608 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7800bc-0b25-4037-ae9a-3789673e7989 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:28.687105 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7800bc-0b25-4037-ae9a-3789673e7989 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:28.687485 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7800bc-0b25-4037-ae9a-3789673e7989 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:28.689185 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 441/873] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:06:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:06:28.694545 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3199015-36fe-4a18-a97d-cf0054be898d None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:28.694973 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3199015-36fe-4a18-a97d-cf0054be898d None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:28.695354 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3199015-36fe-4a18-a97d-cf0054be898d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:28.714233 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-b3199015-36fe-4a18-a97d-cf0054be898d None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:06:28.715510 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-b3199015-36fe-4a18-a97d-cf0054be898d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:06:28.733459 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:28.785729 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 433/874] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:06:28 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 19:06:28.786781 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7800bc-0b25-4037-ae9a-3789673e7989 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:28.894077 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7800bc-0b25-4037-ae9a-3789673e7989 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:28.906375 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:28.931253 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af63faa-f617-4c1b-a9d2-9a9ae75e2141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:28.931917 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af63faa-f617-4c1b-a9d2-9a9ae75e2141 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:28.932198 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af63faa-f617-4c1b-a9d2-9a9ae75e2141 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:28.932473 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af63faa-f617-4c1b-a9d2-9a9ae75e2141 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:28.933060 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af63faa-f617-4c1b-a9d2-9a9ae75e2141 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:28.934495 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af63faa-f617-4c1b-a9d2-9a9ae75e2141 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:28.944039 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-6af63faa-f617-4c1b-a9d2-9a9ae75e2141 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 22 19:06:28.944039 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:28.944039 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 19:06:28.944039 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 19:06:28.944039 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 19:06:28.944039 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 19:06:28.944039 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 19:06:28.944039 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 19:06:28.944039 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 22 19:06:28.944039 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:06:28.944039 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 19:06:28.944039 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:06:28.944039 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:28.944039 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:28.944039 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:28.944039 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:28.944039 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:28.945038 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:28.945038 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:28.945038 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:28.945038 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:28.945038 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:28.945038 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:28.945038 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 19:06:28.945038 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 19:06:28.945038 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 19:06:28.945038 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 19:06:28.945038 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:28.945038 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:28.945038 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:28.945038 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:28.945038 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:28.945038 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:28.945906 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:28.945906 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:28.945906 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:28.945906 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:28.945906 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:28.945906 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:28.945906 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:28.945906 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:28.945906 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:28.945906 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:28.945906 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 19:06:28.945906 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 19:06:28.945906 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 22 19:06:28.945906 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:06:28.950044 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 442/875] 10.222.0.61 () {56 vars in 1126 bytes} [Mon Aug 22 19:06:28 2022] GET /identity/v3/services/object-store => generated 93 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 19:06:28.952822 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3199015-36fe-4a18-a97d-cf0054be898d None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:29.037497 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1239ef91-8e9f-43e9-a3dd-e1f4b04bfac8 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:29.037779 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1239ef91-8e9f-43e9-a3dd-e1f4b04bfac8 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:29.038079 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1239ef91-8e9f-43e9-a3dd-e1f4b04bfac8 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:29.039578 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 443/876] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:06:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:06:29.044641 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef1637f-9d62-4c95-abb8-5e31b82b524e None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:29.044919 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef1637f-9d62-4c95-abb8-5e31b82b524e None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:29.045185 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef1637f-9d62-4c95-abb8-5e31b82b524e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:29.064153 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-5ef1637f-9d62-4c95-abb8-5e31b82b524e None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:06:29.065343 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-5ef1637f-9d62-4c95-abb8-5e31b82b524e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:06:29.066514 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3199015-36fe-4a18-a97d-cf0054be898d None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:29.102790 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed51edb-7afe-4c71-be62-6da273a00afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:29.103764 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed51edb-7afe-4c71-be62-6da273a00afe None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:29.104047 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed51edb-7afe-4c71-be62-6da273a00afe None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:29.104322 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed51edb-7afe-4c71-be62-6da273a00afe None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:29.104982 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed51edb-7afe-4c71-be62-6da273a00afe None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:29.105289 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed51edb-7afe-4c71-be62-6da273a00afe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:29.106711 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed51edb-7afe-4c71-be62-6da273a00afe None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:29.116336 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 434/877] 10.222.0.61 () {56 vars in 1122 bytes} [Mon Aug 22 19:06:28 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:29.116336 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed51edb-7afe-4c71-be62-6da273a00afe None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:29.135993 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:29.148749 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 444/878] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:06:29 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 19:06:29.149878 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef1637f-9d62-4c95-abb8-5e31b82b524e None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:29.226569 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed51edb-7afe-4c71-be62-6da273a00afe None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:29.266702 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6595305-4e62-4f87-a0d1-c11f60eebd80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:29.267716 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6595305-4e62-4f87-a0d1-c11f60eebd80 None admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:29.268216 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6595305-4e62-4f87-a0d1-c11f60eebd80 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:29.268681 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6595305-4e62-4f87-a0d1-c11f60eebd80 None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:29.269514 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6595305-4e62-4f87-a0d1-c11f60eebd80 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:29.271061 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6595305-4e62-4f87-a0d1-c11f60eebd80 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:29.287369 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef1637f-9d62-4c95-abb8-5e31b82b524e None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:29.290118 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:29.291807 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 435/879] 10.222.0.61 () {60 vars in 1141 bytes} [Mon Aug 22 19:06:29 2022] POST /identity/v3/services => generated 251 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:06:29.342762 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d55da27-ca21-4366-9f77-2b2e7ae3bc8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:29.343885 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d55da27-ca21-4366-9f77-2b2e7ae3bc8e None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:29.344328 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d55da27-ca21-4366-9f77-2b2e7ae3bc8e None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:29.344739 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d55da27-ca21-4366-9f77-2b2e7ae3bc8e None admin] PATH_INFO: `/v3/services` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:29.345621 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d55da27-ca21-4366-9f77-2b2e7ae3bc8e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:29.346071 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d55da27-ca21-4366-9f77-2b2e7ae3bc8e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:29.347626 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d55da27-ca21-4366-9f77-2b2e7ae3bc8e None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:29.354109 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 445/880] 10.222.0.61 () {56 vars in 1122 bytes} [Mon Aug 22 19:06:29 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:29.442550 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6595305-4e62-4f87-a0d1-c11f60eebd80 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:29.535018 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:29.564262 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6595305-4e62-4f87-a0d1-c11f60eebd80 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:29.600537 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b11aae4-53e6-4177-bad2-2020ef3629f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:29.601145 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b11aae4-53e6-4177-bad2-2020ef3629f4 None admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:29.601418 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b11aae4-53e6-4177-bad2-2020ef3629f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:29.601687 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b11aae4-53e6-4177-bad2-2020ef3629f4 None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:29.602337 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b11aae4-53e6-4177-bad2-2020ef3629f4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:29.603723 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b11aae4-53e6-4177-bad2-2020ef3629f4 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:29.623638 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 436/881] 10.222.0.61 () {60 vars in 1141 bytes} [Mon Aug 22 19:06:29 2022] POST /identity/v3/services => generated 269 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:06:29.957594 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb4181d-d887-455d-b038-7c33f23e193d None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:29.958130 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb4181d-d887-455d-b038-7c33f23e193d None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:29.958130 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb4181d-d887-455d-b038-7c33f23e193d None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:29.959753 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 446/882] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:06:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:06:29.964984 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ac808f-3f92-407c-abfe-23a2ad818a63 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:29.965282 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ac808f-3f92-407c-abfe-23a2ad818a63 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:29.965553 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ac808f-3f92-407c-abfe-23a2ad818a63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:29.984674 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-a9ac808f-3f92-407c-abfe-23a2ad818a63 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:06:29.985870 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-a9ac808f-3f92-407c-abfe-23a2ad818a63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:06:30.062636 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 437/883] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:06:29 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 19:06:30.327851 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb4181d-d887-455d-b038-7c33f23e193d None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:30.431108 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb4181d-d887-455d-b038-7c33f23e193d None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:30.471390 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ca4515-5132-41e6-8e12-4666d5c0b375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:30.472068 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ca4515-5132-41e6-8e12-4666d5c0b375 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:30.472484 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ca4515-5132-41e6-8e12-4666d5c0b375 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:30.472749 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ca4515-5132-41e6-8e12-4666d5c0b375 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:30.483222 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ca4515-5132-41e6-8e12-4666d5c0b375 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 19:06:30.488334 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-b5ca4515-5132-41e6-8e12-4666d5c0b375 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 22 19:06:30.488334 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:30.488334 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:30.488334 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:30.488334 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:30.488334 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:30.488334 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:30.488334 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:30.488334 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:30.488334 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:30.488334 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:30.488334 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:30.488334 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 19:06:30.488334 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 19:06:30.488334 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 19:06:30.488334 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 19:06:30.489617 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 19:06:30.489617 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 19:06:30.489617 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 19:06:30.489617 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 19:06:30.489617 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:30.489617 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:30.489617 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:30.489617 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:30.489617 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:30.489617 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:30.489617 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:30.489617 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:30.489617 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:30.489617 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:30.489617 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:30.489617 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:30.490466 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:30.490466 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:30.490466 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:30.490466 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:30.490466 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 19:06:30.490466 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 19:06:30.490466 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 19:06:30.490466 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 19:06:30.490466 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 19:06:30.490466 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 19:06:30.490466 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 22 19:06:30.490466 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:06:30.490466 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 447/884] 10.222.0.61 () {56 vars in 1120 bytes} [Mon Aug 22 19:06:30 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:30.498206 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ac808f-3f92-407c-abfe-23a2ad818a63 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:30.567600 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:30.609817 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ac808f-3f92-407c-abfe-23a2ad818a63 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:30.645605 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba25d47-08b1-474d-8ae8-2e36ae4a7160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:30.646569 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba25d47-08b1-474d-8ae8-2e36ae4a7160 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:30.647051 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba25d47-08b1-474d-8ae8-2e36ae4a7160 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:30.647532 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba25d47-08b1-474d-8ae8-2e36ae4a7160 None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:30.648422 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba25d47-08b1-474d-8ae8-2e36ae4a7160 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:30.648923 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba25d47-08b1-474d-8ae8-2e36ae4a7160 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:30.650552 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba25d47-08b1-474d-8ae8-2e36ae4a7160 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:30.659252 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 438/885] 10.222.0.61 () {56 vars in 1115 bytes} [Mon Aug 22 19:06:30 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:30.668724 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ca4515-5132-41e6-8e12-4666d5c0b375 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:30.696043 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:30.778231 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ca4515-5132-41e6-8e12-4666d5c0b375 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:30.813302 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e044ad2c-eeec-4011-86ef-2ea1f698821e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:30.813935 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e044ad2c-eeec-4011-86ef-2ea1f698821e None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:30.814211 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e044ad2c-eeec-4011-86ef-2ea1f698821e None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:30.814480 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e044ad2c-eeec-4011-86ef-2ea1f698821e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:30.825287 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e044ad2c-eeec-4011-86ef-2ea1f698821e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:30.826726 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e044ad2c-eeec-4011-86ef-2ea1f698821e None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:30.840762 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-e044ad2c-eeec-4011-86ef-2ea1f698821e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:06:30.840762 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:30.840762 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:30.840762 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:30.840762 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:30.840762 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:30.840762 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:30.840762 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:30.840762 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:30.840762 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:30.840762 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:30.840762 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:30.840762 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 19:06:30.840762 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 19:06:30.840762 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 19:06:30.840762 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 19:06:30.841823 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:30.841823 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:30.841823 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:30.841823 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:30.841823 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:30.841823 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:30.841823 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:30.841823 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:30.841823 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:30.841823 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:30.841823 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:30.841823 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:30.841823 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:30.841823 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:30.841823 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:30.841823 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:30.841823 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 19:06:30.842778 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 19:06:30.842778 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:06:30.842778 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:06:30.842778 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 448/886] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:06:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:30.847728 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba25d47-08b1-474d-8ae8-2e36ae4a7160 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:30.953393 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba25d47-08b1-474d-8ae8-2e36ae4a7160 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:30.978852 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:30.991817 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325ee90e-3d8d-440a-b9f0-b80d7bd41b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:30.992710 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325ee90e-3d8d-440a-b9f0-b80d7bd41b30 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:30.993217 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325ee90e-3d8d-440a-b9f0-b80d7bd41b30 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:30.993704 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325ee90e-3d8d-440a-b9f0-b80d7bd41b30 None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:30.994546 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325ee90e-3d8d-440a-b9f0-b80d7bd41b30 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:30.995047 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325ee90e-3d8d-440a-b9f0-b80d7bd41b30 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:30.996708 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325ee90e-3d8d-440a-b9f0-b80d7bd41b30 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:31.010151 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 439/887] 10.222.0.61 () {56 vars in 1109 bytes} [Mon Aug 22 19:06:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:31.016062 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e044ad2c-eeec-4011-86ef-2ea1f698821e None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:31.096268 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:31.123474 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e044ad2c-eeec-4011-86ef-2ea1f698821e None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:31.158616 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc86046-0bcf-48bc-bbfe-2eeaf9699af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:31.159248 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc86046-0bcf-48bc-bbfe-2eeaf9699af0 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:31.159571 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc86046-0bcf-48bc-bbfe-2eeaf9699af0 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:31.159883 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc86046-0bcf-48bc-bbfe-2eeaf9699af0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:31.168004 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc86046-0bcf-48bc-bbfe-2eeaf9699af0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:31.169449 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc86046-0bcf-48bc-bbfe-2eeaf9699af0 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:31.181549 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-adc86046-0bcf-48bc-bbfe-2eeaf9699af0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 22 19:06:31.181549 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:31.181549 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:31.181549 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:31.181549 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:31.181549 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:31.181549 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:31.181549 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:31.181549 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:31.181549 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:31.181549 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:31.181549 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:31.181549 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 19:06:31.181549 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 19:06:31.181549 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 19:06:31.181549 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 19:06:31.182525 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:31.182525 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:31.182525 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 19:06:31.182525 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 19:06:31.182525 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 19:06:31.182525 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 19:06:31.182525 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:31.182525 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:31.182525 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:31.182525 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:31.182525 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:31.182525 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:31.182525 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:31.182525 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:31.182525 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:31.182525 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:31.182525 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:31.182525 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:31.184146 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:31.184146 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:31.184146 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 19:06:31.184146 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 19:06:31.184146 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 19:06:31.184146 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 19:06:31.184146 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 19:06:31.184146 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 19:06:31.184146 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 22 19:06:31.184146 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:06:31.184146 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 449/888] 10.222.0.61 () {56 vars in 1117 bytes} [Mon Aug 22 19:06:31 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:31.188733 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325ee90e-3d8d-440a-b9f0-b80d7bd41b30 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:31.298092 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325ee90e-3d8d-440a-b9f0-b80d7bd41b30 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:31.335002 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd08ced5-fc7b-42ed-bcdd-c00322bee42b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:31.335613 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd08ced5-fc7b-42ed-bcdd-c00322bee42b None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:31.335921 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd08ced5-fc7b-42ed-bcdd-c00322bee42b None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:31.336188 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd08ced5-fc7b-42ed-bcdd-c00322bee42b None admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:31.336928 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd08ced5-fc7b-42ed-bcdd-c00322bee42b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:31.337227 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd08ced5-fc7b-42ed-bcdd-c00322bee42b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:31.338639 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd08ced5-fc7b-42ed-bcdd-c00322bee42b None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:31.342290 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cb25a9-bc1b-48a2-a24d-53368934da87 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:31.343371 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cb25a9-bc1b-48a2-a24d-53368934da87 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:31.343891 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cb25a9-bc1b-48a2-a24d-53368934da87 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:31.349286 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 450/889] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:06:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:06:31.351199 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23383f6-6c60-4e01-ab78-c7b9edada214 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:31.351482 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23383f6-6c60-4e01-ab78-c7b9edada214 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:31.351778 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23383f6-6c60-4e01-ab78-c7b9edada214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:31.358985 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 440/890] 10.222.0.61 () {56 vars in 1149 bytes} [Mon Aug 22 19:06:31 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:31.364794 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd08ced5-fc7b-42ed-bcdd-c00322bee42b None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:31.378824 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-b23383f6-6c60-4e01-ab78-c7b9edada214 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:06:31.380460 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-b23383f6-6c60-4e01-ab78-c7b9edada214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:06:31.411234 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:31.459189 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 451/891] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:06:31 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 19:06:31.460464 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef62b261-2580-442c-bf88-f3d6aec00d9c None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:31.460742 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef62b261-2580-442c-bf88-f3d6aec00d9c None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:31.461009 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef62b261-2580-442c-bf88-f3d6aec00d9c None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:31.463675 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 452/892] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:06:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:06:31.467467 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43a030e-343c-4145-9cbc-a5b970694075 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:31.468340 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43a030e-343c-4145-9cbc-a5b970694075 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:31.468619 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43a030e-343c-4145-9cbc-a5b970694075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:31.480216 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd08ced5-fc7b-42ed-bcdd-c00322bee42b None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:31.488562 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-e43a030e-343c-4145-9cbc-a5b970694075 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:06:31.490028 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-e43a030e-343c-4145-9cbc-a5b970694075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:06:31.518411 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0a636a-36cc-4e76-93b5-18af2d144bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:31.519060 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0a636a-36cc-4e76-93b5-18af2d144bdc None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:31.519287 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0a636a-36cc-4e76-93b5-18af2d144bdc None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:31.519559 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0a636a-36cc-4e76-93b5-18af2d144bdc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:31.570429 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:31.581569 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 453/893] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:06:31 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 22 19:06:31.601532 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0a636a-36cc-4e76-93b5-18af2d144bdc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:31.603300 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0a636a-36cc-4e76-93b5-18af2d144bdc None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:31.644908 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c317d9-e76d-4bdf-98d8-3df461d748aa None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:31.645233 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c317d9-e76d-4bdf-98d8-3df461d748aa None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:31.645859 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c317d9-e76d-4bdf-98d8-3df461d748aa None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:31.647823 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 454/894] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:06:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 19:06:31.649831 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:31.652554 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c876f8-0554-476b-a5cd-cb431c623bb8 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:31.652833 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c876f8-0554-476b-a5cd-cb431c623bb8 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:31.653105 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c876f8-0554-476b-a5cd-cb431c623bb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:31.676877 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-44c876f8-0554-476b-a5cd-cb431c623bb8 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:06:31.678413 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-44c876f8-0554-476b-a5cd-cb431c623bb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:06:31.713664 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 441/895] 10.222.0.61 () {58 vars in 1282 bytes} [Mon Aug 22 19:06:31 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 22 19:06:31.719474 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0a636a-36cc-4e76-93b5-18af2d144bdc None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:31.768962 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 455/896] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:06:31 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 19:06:31.771913 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c876f8-0554-476b-a5cd-cb431c623bb8 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:31.834267 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0a636a-36cc-4e76-93b5-18af2d144bdc None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:31.870138 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62eec038-b090-4b4e-8cac-d28c94e1c6a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:31.870928 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eec038-b090-4b4e-8cac-d28c94e1c6a0 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:31.871509 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eec038-b090-4b4e-8cac-d28c94e1c6a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:31.871869 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eec038-b090-4b4e-8cac-d28c94e1c6a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:31.877633 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c876f8-0554-476b-a5cd-cb431c623bb8 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:31.882619 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62eec038-b090-4b4e-8cac-d28c94e1c6a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:31.884083 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62eec038-b090-4b4e-8cac-d28c94e1c6a0 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:31.885794 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:31.901325 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-62eec038-b090-4b4e-8cac-d28c94e1c6a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:06:31.901325 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:31.901325 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:31.901325 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:31.901325 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:31.901325 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:31.901325 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:31.901325 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:31.901325 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:31.901325 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:31.901325 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:31.901325 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:31.901325 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 19:06:31.901325 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 19:06:31.901325 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 19:06:31.901325 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 19:06:31.902322 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:31.902322 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:31.902322 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:31.902322 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:31.902322 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:31.902322 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:31.902322 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:31.902322 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:31.902322 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:31.902322 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:31.902322 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:31.902322 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:31.902322 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:31.902322 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:31.902322 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:31.902322 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:31.902322 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 19:06:31.903223 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 19:06:31.903223 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:06:31.903223 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:06:31.903223 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 442/897] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:06:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:31.903223 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62eec038-b090-4b4e-8cac-d28c94e1c6a0 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:31.910429 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:31.919849 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7992949-2f1a-4e2b-9856-c50cc2a6afae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:31.920460 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7992949-2f1a-4e2b-9856-c50cc2a6afae None admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:31.920751 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7992949-2f1a-4e2b-9856-c50cc2a6afae None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:31.921023 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7992949-2f1a-4e2b-9856-c50cc2a6afae None admin] PATH_INFO: `/v3/services` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:31.921687 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7992949-2f1a-4e2b-9856-c50cc2a6afae None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:31.923016 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7992949-2f1a-4e2b-9856-c50cc2a6afae None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:31.939182 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 456/898] 10.222.0.61 () {60 vars in 1141 bytes} [Mon Aug 22 19:06:31 2022] POST /identity/v3/services => generated 241 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 19:06:31.940620 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7992949-2f1a-4e2b-9856-c50cc2a6afae None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:32.023537 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62eec038-b090-4b4e-8cac-d28c94e1c6a0 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:32.068668 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6628598e-6c0f-4095-8448-4e8b77ef1267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:32.069424 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6628598e-6c0f-4095-8448-4e8b77ef1267 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:32.069732 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6628598e-6c0f-4095-8448-4e8b77ef1267 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:32.070084 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6628598e-6c0f-4095-8448-4e8b77ef1267 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:32.070676 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6628598e-6c0f-4095-8448-4e8b77ef1267 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:32.072088 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6628598e-6c0f-4095-8448-4e8b77ef1267 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:32.074233 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7992949-2f1a-4e2b-9856-c50cc2a6afae None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:32.082548 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-6628598e-6c0f-4095-8448-4e8b77ef1267 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 22 19:06:32.082548 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:32.082548 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 19:06:32.082548 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 19:06:32.082548 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 19:06:32.082548 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 19:06:32.082548 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 19:06:32.082548 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 19:06:32.082548 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 22 19:06:32.082548 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:06:32.082548 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 19:06:32.082548 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:06:32.082548 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:32.082548 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:32.082548 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:32.082548 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:32.082548 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:32.083916 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:32.083916 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:32.083916 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:32.083916 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:32.083916 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:32.083916 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:32.083916 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 19:06:32.083916 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 19:06:32.083916 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 19:06:32.083916 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 19:06:32.083916 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:32.083916 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:32.083916 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:32.083916 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:32.083916 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:32.083916 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:32.085001 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:32.085001 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:32.085001 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:32.085001 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:32.085001 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:32.085001 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:32.085001 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:32.085001 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:32.085001 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:32.085001 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:32.085001 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 19:06:32.085001 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 19:06:32.085001 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 22 19:06:32.085001 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:06:32.085001 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 443/899] 10.222.0.61 () {56 vars in 1114 bytes} [Mon Aug 22 19:06:31 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:32.085988 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6628598e-6c0f-4095-8448-4e8b77ef1267 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:32.123513 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044c47d9-5c13-42d3-9ffc-964f261e3dbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:32.125141 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044c47d9-5c13-42d3-9ffc-964f261e3dbf None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:32.125141 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044c47d9-5c13-42d3-9ffc-964f261e3dbf None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:32.125314 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044c47d9-5c13-42d3-9ffc-964f261e3dbf None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:32.125888 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044c47d9-5c13-42d3-9ffc-964f261e3dbf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:32.126184 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044c47d9-5c13-42d3-9ffc-964f261e3dbf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:32.127872 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044c47d9-5c13-42d3-9ffc-964f261e3dbf None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:32.140902 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 457/900] 10.222.0.61 () {56 vars in 1109 bytes} [Mon Aug 22 19:06:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:32.141745 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044c47d9-5c13-42d3-9ffc-964f261e3dbf None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:32.160083 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:32.187576 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:32.209860 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6628598e-6c0f-4095-8448-4e8b77ef1267 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:32.247730 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2abac4-f292-4fd7-a369-2d7d08e87da8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:32.248404 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2abac4-f292-4fd7-a369-2d7d08e87da8 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:32.248668 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2abac4-f292-4fd7-a369-2d7d08e87da8 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:32.248968 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2abac4-f292-4fd7-a369-2d7d08e87da8 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:32.249551 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2abac4-f292-4fd7-a369-2d7d08e87da8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:32.250130 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044c47d9-5c13-42d3-9ffc-964f261e3dbf None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:32.250907 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2abac4-f292-4fd7-a369-2d7d08e87da8 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:32.262153 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-fa2abac4-f292-4fd7-a369-2d7d08e87da8 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 22 19:06:32.262153 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:32.262153 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 19:06:32.262153 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 19:06:32.262153 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 19:06:32.262153 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 19:06:32.262153 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 19:06:32.262153 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 19:06:32.262153 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 22 19:06:32.262153 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:06:32.262153 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 19:06:32.262153 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:06:32.262153 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:32.262153 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:32.262153 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:32.262153 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:32.263111 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:32.263111 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:32.263111 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:32.263111 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:32.263111 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:32.263111 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:32.263111 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:32.263111 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 19:06:32.263111 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 19:06:32.263111 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 19:06:32.263111 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 19:06:32.263111 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:32.263111 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:32.263111 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:32.263111 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:32.263111 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:32.263111 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:32.268555 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:32.268555 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:32.268555 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:32.268555 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:32.268555 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:32.268555 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:32.268555 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:32.268555 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:32.268555 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:32.268555 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:32.268555 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 19:06:32.268555 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 19:06:32.268555 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 22 19:06:32.268555 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:06:32.268555 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 444/901] 10.222.0.61 () {56 vars in 1132 bytes} [Mon Aug 22 19:06:32 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:32.269838 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2abac4-f292-4fd7-a369-2d7d08e87da8 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:32.286822 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f15bc9-25f6-4d83-bee1-ac4132f0a8c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:32.287446 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f15bc9-25f6-4d83-bee1-ac4132f0a8c0 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:32.287739 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f15bc9-25f6-4d83-bee1-ac4132f0a8c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:32.288015 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f15bc9-25f6-4d83-bee1-ac4132f0a8c0 None admin] PATH_INFO: `/v3/services` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:32.288644 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f15bc9-25f6-4d83-bee1-ac4132f0a8c0 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:32.288943 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f15bc9-25f6-4d83-bee1-ac4132f0a8c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:32.290299 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f15bc9-25f6-4d83-bee1-ac4132f0a8c0 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:32.295300 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 458/902] 10.222.0.61 () {56 vars in 1114 bytes} [Mon Aug 22 19:06:32 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:32.300489 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f15bc9-25f6-4d83-bee1-ac4132f0a8c0 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:32.374387 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2abac4-f292-4fd7-a369-2d7d08e87da8 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:32.392523 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:32.409901 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18f32dd-6aa5-4a79-be50-f29cc2e2ebc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:32.410535 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18f32dd-6aa5-4a79-be50-f29cc2e2ebc4 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:32.410812 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18f32dd-6aa5-4a79-be50-f29cc2e2ebc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:32.411080 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18f32dd-6aa5-4a79-be50-f29cc2e2ebc4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:32.415476 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f15bc9-25f6-4d83-bee1-ac4132f0a8c0 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:32.417887 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18f32dd-6aa5-4a79-be50-f29cc2e2ebc4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:32.419330 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18f32dd-6aa5-4a79-be50-f29cc2e2ebc4 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:32.424340 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:32.430825 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-d18f32dd-6aa5-4a79-be50-f29cc2e2ebc4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 19:06:32.430825 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:32.430825 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:32.430825 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:32.430825 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:32.430825 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:32.430825 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:32.430825 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:32.430825 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:32.430825 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:32.430825 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:32.430825 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:32.430825 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 19:06:32.430825 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 19:06:32.430825 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 19:06:32.431788 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 19:06:32.431788 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:32.431788 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:32.431788 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:32.431788 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:32.431788 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:32.431788 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:32.431788 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:32.431788 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:32.431788 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:32.431788 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:32.431788 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:32.431788 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:32.431788 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:32.431788 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:32.431788 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:32.431788 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:32.432702 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 19:06:32.432702 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 19:06:32.432702 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 19:06:32.432702 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 19:06:32.432702 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 19:06:32.432702 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 19:06:32.432702 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 19:06:32.432702 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:06:32.432702 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 445/903] 10.222.0.61 () {56 vars in 1111 bytes} [Mon Aug 22 19:06:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:32.432702 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18f32dd-6aa5-4a79-be50-f29cc2e2ebc4 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:32.458616 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7fe295-64dc-491b-a43c-556ae96c9a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:32.459234 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7fe295-64dc-491b-a43c-556ae96c9a5f None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:32.459511 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7fe295-64dc-491b-a43c-556ae96c9a5f None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:32.459816 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7fe295-64dc-491b-a43c-556ae96c9a5f None admin] PATH_INFO: `/v3/services` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:32.460449 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7fe295-64dc-491b-a43c-556ae96c9a5f None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:32.461032 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7fe295-64dc-491b-a43c-556ae96c9a5f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:32.462111 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7fe295-64dc-491b-a43c-556ae96c9a5f None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:32.467075 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 459/904] 10.222.0.61 () {56 vars in 1126 bytes} [Mon Aug 22 19:06:32 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:32.468263 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7fe295-64dc-491b-a43c-556ae96c9a5f None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:32.546445 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18f32dd-6aa5-4a79-be50-f29cc2e2ebc4 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:32.576393 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7fe295-64dc-491b-a43c-556ae96c9a5f None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:32.582046 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f63251b-f792-49e1-bf34-1113a87434bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:32.582657 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f63251b-f792-49e1-bf34-1113a87434bb None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:32.582934 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f63251b-f792-49e1-bf34-1113a87434bb None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:32.583206 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f63251b-f792-49e1-bf34-1113a87434bb None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:32.583866 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f63251b-f792-49e1-bf34-1113a87434bb None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:32.584168 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f63251b-f792-49e1-bf34-1113a87434bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:32.585564 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f63251b-f792-49e1-bf34-1113a87434bb None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:32.595201 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 446/905] 10.222.0.61 () {56 vars in 1114 bytes} [Mon Aug 22 19:06:32 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:32.595201 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f63251b-f792-49e1-bf34-1113a87434bb None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:32.625948 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e4199a-79e3-4711-877a-91ba5ee03e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:32.626579 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e4199a-79e3-4711-877a-91ba5ee03e5a None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:32.626861 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e4199a-79e3-4711-877a-91ba5ee03e5a None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:32.627133 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e4199a-79e3-4711-877a-91ba5ee03e5a None admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:32.627856 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e4199a-79e3-4711-877a-91ba5ee03e5a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:32.628166 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e4199a-79e3-4711-877a-91ba5ee03e5a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:32.629582 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e4199a-79e3-4711-877a-91ba5ee03e5a None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:32.646389 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 460/906] 10.222.0.61 () {56 vars in 1148 bytes} [Mon Aug 22 19:06:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:32.646389 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e4199a-79e3-4711-877a-91ba5ee03e5a None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:32.649008 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:32.672868 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:32.710067 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f63251b-f792-49e1-bf34-1113a87434bb None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:32.743998 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e4199a-79e3-4711-877a-91ba5ee03e5a None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:32.744167 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c3281d-6493-4345-8262-ea3f9632a11e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:32.744720 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c3281d-6493-4345-8262-ea3f9632a11e None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:32.744997 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c3281d-6493-4345-8262-ea3f9632a11e None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:32.745264 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c3281d-6493-4345-8262-ea3f9632a11e None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:32.745893 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c3281d-6493-4345-8262-ea3f9632a11e None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:32.746191 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c3281d-6493-4345-8262-ea3f9632a11e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:32.747546 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c3281d-6493-4345-8262-ea3f9632a11e None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:32.753065 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 447/907] 10.222.0.61 () {56 vars in 1126 bytes} [Mon Aug 22 19:06:32 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:32.758388 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c3281d-6493-4345-8262-ea3f9632a11e None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:32.779923 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d83ec2-1da2-43d3-8d3a-a94335983137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:32.780972 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d83ec2-1da2-43d3-8d3a-a94335983137 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:32.781257 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d83ec2-1da2-43d3-8d3a-a94335983137 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:32.781533 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d83ec2-1da2-43d3-8d3a-a94335983137 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:32.782231 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d83ec2-1da2-43d3-8d3a-a94335983137 None admin] RBAC: Adding query filter params (interface=public, service_id=60a108c9339044c8a5a1276287b458bd, region_id=RegionOne) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:32.782565 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d83ec2-1da2-43d3-8d3a-a94335983137 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:32.783931 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d83ec2-1da2-43d3-8d3a-a94335983137 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:32.789049 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 461/908] 10.222.0.61 () {56 vars in 1251 bytes} [Mon Aug 22 19:06:32 2022] GET /identity/v3/endpoints?service_id=60a108c9339044c8a5a1276287b458bd&interface=public®ion_id=RegionOne => generated 194 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:32.789820 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d83ec2-1da2-43d3-8d3a-a94335983137 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:32.870879 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c3281d-6493-4345-8262-ea3f9632a11e None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:32.878602 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:32.889376 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:32.897776 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d83ec2-1da2-43d3-8d3a-a94335983137 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:32.909260 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfe25d3-6a6b-43de-aafe-0e57733bdcae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:32.910013 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfe25d3-6a6b-43de-aafe-0e57733bdcae None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:32.910299 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfe25d3-6a6b-43de-aafe-0e57733bdcae None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:32.910567 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfe25d3-6a6b-43de-aafe-0e57733bdcae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:32.911301 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfe25d3-6a6b-43de-aafe-0e57733bdcae None admin] RBAC: Adding query filter params (role.id=aa9eb04acf884c51ad2adb05380bc7a1, scope.project.id=e07dc5ebdc504c6da706a2d21c0a3970, user.id=399396a8fe254da199bc48b0ca5243d0) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:32.911616 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfe25d3-6a6b-43de-aafe-0e57733bdcae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:32.913066 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfe25d3-6a6b-43de-aafe-0e57733bdcae None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:32.928043 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 448/909] 10.222.0.61 () {56 vars in 1374 bytes} [Mon Aug 22 19:06:32 2022] GET /identity/v3/role_assignments?user.id=399396a8fe254da199bc48b0ca5243d0&scope.project.id=e07dc5ebdc504c6da706a2d21c0a3970&role.id=aa9eb04acf884c51ad2adb05380bc7a1 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:32.928858 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfe25d3-6a6b-43de-aafe-0e57733bdcae None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:32.939250 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f0e90f-0f43-47fd-8d7a-1771198a9bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:32.939914 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f0e90f-0f43-47fd-8d7a-1771198a9bc6 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:32.940193 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f0e90f-0f43-47fd-8d7a-1771198a9bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:32.943699 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f0e90f-0f43-47fd-8d7a-1771198a9bc6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:32.944541 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f0e90f-0f43-47fd-8d7a-1771198a9bc6 None admin] RBAC: Adding query filter params (interface=public, service_id=a34c9a740cdd465ea8361823d0b0e9d7, region_id=RegionOne) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:32.944857 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f0e90f-0f43-47fd-8d7a-1771198a9bc6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:32.946403 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f0e90f-0f43-47fd-8d7a-1771198a9bc6 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:32.954685 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 462/910] 10.222.0.61 () {56 vars in 1251 bytes} [Mon Aug 22 19:06:32 2022] GET /identity/v3/endpoints?service_id=a34c9a740cdd465ea8361823d0b0e9d7&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:32.960407 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f0e90f-0f43-47fd-8d7a-1771198a9bc6 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:33.062468 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfe25d3-6a6b-43de-aafe-0e57733bdcae None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:33.097630 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f0e90f-0f43-47fd-8d7a-1771198a9bc6 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:33.111514 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8463e860-8c37-4e3d-9390-6280d2f20984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:33.115217 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8463e860-8c37-4e3d-9390-6280d2f20984 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:33.115217 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8463e860-8c37-4e3d-9390-6280d2f20984 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:33.115217 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8463e860-8c37-4e3d-9390-6280d2f20984 None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:33.115217 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8463e860-8c37-4e3d-9390-6280d2f20984 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:33.115217 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8463e860-8c37-4e3d-9390-6280d2f20984 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:33.127678 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 449/911] 10.222.0.61 () {56 vars in 1087 bytes} [Mon Aug 22 19:06:32 2022] GET /identity/v3/services => generated 1497 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:06:33.157128 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c392d74d-48dd-4966-ba19-d26a5c236ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:33.157829 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c392d74d-48dd-4966-ba19-d26a5c236ab0 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:33.158107 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c392d74d-48dd-4966-ba19-d26a5c236ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:33.158423 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c392d74d-48dd-4966-ba19-d26a5c236ab0 None admin] PATH_INFO: `/v3/services` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:33.159050 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c392d74d-48dd-4966-ba19-d26a5c236ab0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:33.160484 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c392d74d-48dd-4966-ba19-d26a5c236ab0 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:33.167879 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 463/912] 10.222.0.61 () {56 vars in 1087 bytes} [Mon Aug 22 19:06:32 2022] GET /identity/v3/services => generated 1497 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 19:06:33.886942 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa81755-7c43-448f-b2de-8e1900114ee7 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:33.887476 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa81755-7c43-448f-b2de-8e1900114ee7 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:33.887476 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa81755-7c43-448f-b2de-8e1900114ee7 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:33.889075 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 450/913] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:06:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:06:33.894469 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e719d10d-10a1-4233-be0c-ed69661f1060 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:33.894758 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e719d10d-10a1-4233-be0c-ed69661f1060 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:33.895028 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e719d10d-10a1-4233-be0c-ed69661f1060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:33.913188 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-e719d10d-10a1-4233-be0c-ed69661f1060 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:06:33.914377 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-e719d10d-10a1-4233-be0c-ed69661f1060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:06:33.932205 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:33.985354 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 464/914] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:06:33 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 19:06:34.252858 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa81755-7c43-448f-b2de-8e1900114ee7 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:34.290586 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:34.370824 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa81755-7c43-448f-b2de-8e1900114ee7 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:34.407373 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426d1c96-513b-4434-a83d-f45c61c6473c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:34.408301 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426d1c96-513b-4434-a83d-f45c61c6473c None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:34.408728 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426d1c96-513b-4434-a83d-f45c61c6473c None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:34.409171 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426d1c96-513b-4434-a83d-f45c61c6473c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:34.409915 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426d1c96-513b-4434-a83d-f45c61c6473c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:34.411420 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426d1c96-513b-4434-a83d-f45c61c6473c None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:34.421607 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-426d1c96-513b-4434-a83d-f45c61c6473c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 22 19:06:34.421607 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:34.421607 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 19:06:34.421607 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 19:06:34.421607 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 19:06:34.421607 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 19:06:34.421607 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 19:06:34.421607 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 19:06:34.421607 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 22 19:06:34.421607 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:06:34.421607 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 19:06:34.421607 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:06:34.421607 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:34.421607 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:34.421607 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:34.421607 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:34.421607 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:34.422816 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:34.422816 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:34.422816 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:34.422816 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:34.422816 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:34.422816 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:34.422816 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 19:06:34.422816 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 19:06:34.422816 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 19:06:34.422816 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 19:06:34.422816 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:34.422816 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:34.422816 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:34.422816 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:34.422816 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:34.422816 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:34.423877 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:34.423877 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:34.423877 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:34.423877 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:34.423877 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:34.423877 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:34.423877 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:34.423877 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:34.423877 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:34.423877 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:34.423877 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 19:06:34.423877 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 19:06:34.423877 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 22 19:06:34.423877 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:06:34.426187 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 451/915] 10.222.0.61 () {56 vars in 1126 bytes} [Mon Aug 22 19:06:34 2022] GET /identity/v3/services/object-store => generated 93 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:34.434009 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e719d10d-10a1-4233-be0c-ed69661f1060 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:34.533618 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e719d10d-10a1-4233-be0c-ed69661f1060 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:34.569212 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd91df92-485c-4b63-9b12-3a20a73ea5ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:34.569873 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd91df92-485c-4b63-9b12-3a20a73ea5ce None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:34.570153 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd91df92-485c-4b63-9b12-3a20a73ea5ce None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:34.570421 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd91df92-485c-4b63-9b12-3a20a73ea5ce None admin] PATH_INFO: `/v3/services` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:34.571053 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd91df92-485c-4b63-9b12-3a20a73ea5ce None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:34.571351 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd91df92-485c-4b63-9b12-3a20a73ea5ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:34.572744 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd91df92-485c-4b63-9b12-3a20a73ea5ce None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:34.577834 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 465/916] 10.222.0.61 () {56 vars in 1122 bytes} [Mon Aug 22 19:06:34 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:34.583511 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426d1c96-513b-4434-a83d-f45c61c6473c None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:34.688410 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:34.692804 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426d1c96-513b-4434-a83d-f45c61c6473c None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:34.728723 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42df7be6-8cb9-4ec5-b0dd-648791264e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:34.729331 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42df7be6-8cb9-4ec5-b0dd-648791264e6e None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:34.729607 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42df7be6-8cb9-4ec5-b0dd-648791264e6e None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:34.729872 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42df7be6-8cb9-4ec5-b0dd-648791264e6e None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:34.730500 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42df7be6-8cb9-4ec5-b0dd-648791264e6e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:34.730797 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42df7be6-8cb9-4ec5-b0dd-648791264e6e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:34.732189 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42df7be6-8cb9-4ec5-b0dd-648791264e6e None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:34.737936 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 452/917] 10.222.0.61 () {56 vars in 1122 bytes} [Mon Aug 22 19:06:34 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:34.744270 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd91df92-485c-4b63-9b12-3a20a73ea5ce None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:34.763204 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:34.849543 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd91df92-485c-4b63-9b12-3a20a73ea5ce None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:34.853995 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0902922-58e9-42eb-bd54-32fe21dd7169 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:34.854278 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0902922-58e9-42eb-bd54-32fe21dd7169 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:34.854549 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0902922-58e9-42eb-bd54-32fe21dd7169 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:34.856263 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 453/918] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:06:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:06:34.864065 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7720d8-f4ba-4b24-afc6-7fb137c5ee4f None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:34.864455 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7720d8-f4ba-4b24-afc6-7fb137c5ee4f None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:34.864824 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7720d8-f4ba-4b24-afc6-7fb137c5ee4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:34.882948 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-ae7720d8-f4ba-4b24-afc6-7fb137c5ee4f None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:06:34.884318 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-ae7720d8-f4ba-4b24-afc6-7fb137c5ee4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:06:34.888842 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c966d07-4964-4bf7-b5c3-118f5b79a0e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:34.889492 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c966d07-4964-4bf7-b5c3-118f5b79a0e0 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:34.889775 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c966d07-4964-4bf7-b5c3-118f5b79a0e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:34.890052 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c966d07-4964-4bf7-b5c3-118f5b79a0e0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:34.890758 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c966d07-4964-4bf7-b5c3-118f5b79a0e0 None admin] RBAC: Adding query filter params (interface=public, service_id=4983e00fc60c4c0fafb5e2395ad5d267, region_id=RegionOne) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:34.891112 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c966d07-4964-4bf7-b5c3-118f5b79a0e0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:34.892609 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c966d07-4964-4bf7-b5c3-118f5b79a0e0 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:34.898637 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 466/919] 10.222.0.61 () {56 vars in 1251 bytes} [Mon Aug 22 19:06:34 2022] GET /identity/v3/endpoints?service_id=4983e00fc60c4c0fafb5e2395ad5d267&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:34.904382 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c966d07-4964-4bf7-b5c3-118f5b79a0e0 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:34.979836 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 454/920] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:06:34 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 19:06:35.000869 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:35.020629 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c966d07-4964-4bf7-b5c3-118f5b79a0e0 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:35.056267 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ad5ad2-a54e-4ebf-b5f5-974a1c9c4bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:35.056881 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ad5ad2-a54e-4ebf-b5f5-974a1c9c4bbb None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:35.057158 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ad5ad2-a54e-4ebf-b5f5-974a1c9c4bbb None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:35.057431 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ad5ad2-a54e-4ebf-b5f5-974a1c9c4bbb None admin] PATH_INFO: `/v3/services` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:35.058102 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ad5ad2-a54e-4ebf-b5f5-974a1c9c4bbb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:35.059520 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ad5ad2-a54e-4ebf-b5f5-974a1c9c4bbb None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:35.066110 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 467/921] 10.222.0.61 () {56 vars in 1087 bytes} [Mon Aug 22 19:06:34 2022] GET /identity/v3/services => generated 1497 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 19:06:35.077546 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25df3a75-7ffb-422d-b44b-de55fc1ccac9 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:35.077878 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25df3a75-7ffb-422d-b44b-de55fc1ccac9 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:35.078112 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25df3a75-7ffb-422d-b44b-de55fc1ccac9 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:35.079867 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 455/922] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:06:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:06:35.085755 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3033b839-d4d6-40b5-9d83-404d7cc0423a None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:35.086072 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3033b839-d4d6-40b5-9d83-404d7cc0423a None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:35.086306 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3033b839-d4d6-40b5-9d83-404d7cc0423a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:35.106111 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-3033b839-d4d6-40b5-9d83-404d7cc0423a None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:06:35.107327 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-3033b839-d4d6-40b5-9d83-404d7cc0423a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:06:35.195260 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 468/923] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:06:35 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 19:06:35.248160 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7347a275-8922-433b-a219-81d048066039 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:35.248462 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3033b839-d4d6-40b5-9d83-404d7cc0423a None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:35.248652 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7347a275-8922-433b-a219-81d048066039 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:35.248652 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7347a275-8922-433b-a219-81d048066039 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:35.250141 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 456/924] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:06:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:06:35.255948 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c22cfb-da2c-41f4-96b0-2a6e48cd6315 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:35.256231 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c22cfb-da2c-41f4-96b0-2a6e48cd6315 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:35.256504 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c22cfb-da2c-41f4-96b0-2a6e48cd6315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:35.277791 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-41c22cfb-da2c-41f4-96b0-2a6e48cd6315 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:06:35.280894 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-41c22cfb-da2c-41f4-96b0-2a6e48cd6315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:06:35.336068 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:35.364559 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3033b839-d4d6-40b5-9d83-404d7cc0423a None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:35.387670 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 457/925] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:06:35 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 19:06:35.406198 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8171ff3-959a-4398-b9af-7e528b45e081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:35.406839 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8171ff3-959a-4398-b9af-7e528b45e081 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:35.407119 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8171ff3-959a-4398-b9af-7e528b45e081 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:35.407391 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8171ff3-959a-4398-b9af-7e528b45e081 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:35.418322 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8171ff3-959a-4398-b9af-7e528b45e081 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 19:06:35.423198 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-f8171ff3-959a-4398-b9af-7e528b45e081 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 22 19:06:35.423198 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:35.423198 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:35.423198 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:35.423198 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:35.423198 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:35.423198 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:35.423198 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:35.423198 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:35.423198 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:35.423198 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:35.423198 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:35.423198 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 19:06:35.423198 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 19:06:35.423198 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 19:06:35.423198 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 19:06:35.424253 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 19:06:35.424253 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 19:06:35.424253 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 19:06:35.424253 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 19:06:35.424253 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:35.424253 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:35.424253 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:35.424253 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:35.424253 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:35.424253 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:35.424253 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:35.424253 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:35.424253 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:35.424253 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:35.424253 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:35.424253 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:35.425731 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:35.425731 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:35.425731 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:35.425731 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:35.425731 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 19:06:35.425731 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 19:06:35.425731 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 19:06:35.425731 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 19:06:35.425731 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 19:06:35.425731 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 19:06:35.425731 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 22 19:06:35.425731 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:06:35.425731 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 469/926] 10.222.0.61 () {56 vars in 1120 bytes} [Mon Aug 22 19:06:35 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 19:06:35.430308 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c22cfb-da2c-41f4-96b0-2a6e48cd6315 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:35.455683 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:35.468996 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8171ff3-959a-4398-b9af-7e528b45e081 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:35.541411 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c22cfb-da2c-41f4-96b0-2a6e48cd6315 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:35.581263 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfc6949-3cdd-458e-822e-4a4c0c9a4d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:35.581875 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfc6949-3cdd-458e-822e-4a4c0c9a4d18 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:35.582201 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfc6949-3cdd-458e-822e-4a4c0c9a4d18 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:35.582480 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfc6949-3cdd-458e-822e-4a4c0c9a4d18 None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:35.583150 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfc6949-3cdd-458e-822e-4a4c0c9a4d18 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:35.583450 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfc6949-3cdd-458e-822e-4a4c0c9a4d18 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:35.584905 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfc6949-3cdd-458e-822e-4a4c0c9a4d18 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:35.597993 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 458/927] 10.222.0.61 () {56 vars in 1115 bytes} [Mon Aug 22 19:06:35 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:35.599035 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfc6949-3cdd-458e-822e-4a4c0c9a4d18 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:35.604993 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8171ff3-959a-4398-b9af-7e528b45e081 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:35.642187 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a441cb0-5343-4e22-baa9-97ac8b6cb4dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:35.642880 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a441cb0-5343-4e22-baa9-97ac8b6cb4dd None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:35.643098 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a441cb0-5343-4e22-baa9-97ac8b6cb4dd None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:35.643369 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a441cb0-5343-4e22-baa9-97ac8b6cb4dd None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:35.644003 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a441cb0-5343-4e22-baa9-97ac8b6cb4dd None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:35.645426 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a441cb0-5343-4e22-baa9-97ac8b6cb4dd None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:35.654360 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-8a441cb0-5343-4e22-baa9-97ac8b6cb4dd None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 22 19:06:35.654360 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:35.654360 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 19:06:35.654360 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 19:06:35.654360 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 19:06:35.654360 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 19:06:35.654360 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 19:06:35.654360 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 19:06:35.654360 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 22 19:06:35.654360 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:06:35.654360 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 19:06:35.654360 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:06:35.654360 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:35.654360 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:35.654360 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:35.654360 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:35.654360 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:35.655414 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:35.655414 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:35.655414 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:35.655414 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:35.655414 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:35.655414 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:35.655414 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 19:06:35.655414 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 19:06:35.655414 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 19:06:35.655414 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 19:06:35.655414 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:35.655414 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:35.655414 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:35.655414 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:35.655414 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:35.655414 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:35.656278 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:35.656278 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:35.656278 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:35.656278 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:35.656278 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:35.656278 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:35.656278 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:35.656278 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:35.656278 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:35.656278 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:35.656278 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 19:06:35.656278 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 19:06:35.656278 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 22 19:06:35.656278 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:06:35.659922 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 470/928] 10.222.0.61 () {56 vars in 1114 bytes} [Mon Aug 22 19:06:35 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 19:06:35.660762 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:35.665216 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a441cb0-5343-4e22-baa9-97ac8b6cb4dd None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:35.707925 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:35.726408 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfc6949-3cdd-458e-822e-4a4c0c9a4d18 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:35.762284 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514c4da1-068c-493f-a967-25f9c27877ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:35.762922 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514c4da1-068c-493f-a967-25f9c27877ef None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:35.763202 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514c4da1-068c-493f-a967-25f9c27877ef None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:35.763472 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514c4da1-068c-493f-a967-25f9c27877ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:35.777192 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514c4da1-068c-493f-a967-25f9c27877ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:35.777817 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a441cb0-5343-4e22-baa9-97ac8b6cb4dd None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:35.778617 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514c4da1-068c-493f-a967-25f9c27877ef None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:35.793993 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-514c4da1-068c-493f-a967-25f9c27877ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:06:35.793993 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:35.793993 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:35.793993 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:35.793993 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:35.793993 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:35.793993 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:35.793993 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:35.793993 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:35.793993 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:35.793993 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:35.793993 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:35.793993 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 19:06:35.793993 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 19:06:35.793993 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 19:06:35.793993 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 19:06:35.794957 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:35.794957 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:35.794957 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:35.794957 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:35.794957 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:35.794957 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:35.794957 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:35.794957 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:35.794957 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:35.794957 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:35.794957 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:35.794957 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:35.794957 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:35.794957 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:35.794957 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:35.794957 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:35.794957 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 19:06:35.796735 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 19:06:35.796735 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:06:35.796735 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:06:35.796735 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 459/929] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:06:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 19:06:35.796735 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514c4da1-068c-493f-a967-25f9c27877ef None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:35.819239 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06618297-e678-464d-af68-3df1e93c45f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:35.820183 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06618297-e678-464d-af68-3df1e93c45f0 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:35.820460 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06618297-e678-464d-af68-3df1e93c45f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:35.820727 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06618297-e678-464d-af68-3df1e93c45f0 None admin] PATH_INFO: `/v3/services` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:35.821356 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06618297-e678-464d-af68-3df1e93c45f0 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:35.821661 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06618297-e678-464d-af68-3df1e93c45f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:35.823445 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06618297-e678-464d-af68-3df1e93c45f0 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:35.829321 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 471/930] 10.222.0.61 () {56 vars in 1114 bytes} [Mon Aug 22 19:06:35 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 19:06:35.832481 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06618297-e678-464d-af68-3df1e93c45f0 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:35.903160 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:35.905125 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514c4da1-068c-493f-a967-25f9c27877ef None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:35.942039 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28927b2-05ae-4f87-9530-d33b4abee8e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:35.942677 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28927b2-05ae-4f87-9530-d33b4abee8e9 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:35.942955 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28927b2-05ae-4f87-9530-d33b4abee8e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:35.943224 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28927b2-05ae-4f87-9530-d33b4abee8e9 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:35.944205 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28927b2-05ae-4f87-9530-d33b4abee8e9 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:35.945574 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28927b2-05ae-4f87-9530-d33b4abee8e9 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:35.950714 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:35.954545 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06618297-e678-464d-af68-3df1e93c45f0 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:35.960957 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-b28927b2-05ae-4f87-9530-d33b4abee8e9 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 22 19:06:35.960957 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:35.960957 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 19:06:35.960957 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 19:06:35.960957 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 19:06:35.960957 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 19:06:35.960957 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 19:06:35.960957 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 19:06:35.960957 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 22 19:06:35.960957 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:06:35.960957 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 19:06:35.960957 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:06:35.960957 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:35.960957 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:35.960957 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:35.960957 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:35.961923 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:35.961923 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:35.961923 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:35.961923 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:35.961923 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:35.961923 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:35.961923 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:35.961923 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 19:06:35.961923 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 19:06:35.961923 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 19:06:35.961923 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 19:06:35.961923 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:35.961923 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:35.961923 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:35.961923 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:35.961923 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:35.961923 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:35.962833 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:35.962833 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:35.962833 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:35.962833 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:35.962833 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:35.962833 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:35.962833 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:35.962833 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:35.962833 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:35.962833 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:35.962833 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 19:06:35.962833 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 19:06:35.962833 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 22 19:06:35.962833 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:06:35.970198 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 460/931] 10.222.0.61 () {56 vars in 1132 bytes} [Mon Aug 22 19:06:35 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:35.970198 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28927b2-05ae-4f87-9530-d33b4abee8e9 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:36.007865 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca1eb84-6629-47e6-8b46-a25c91d81a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:36.008910 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca1eb84-6629-47e6-8b46-a25c91d81a70 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:36.009385 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca1eb84-6629-47e6-8b46-a25c91d81a70 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:36.009851 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca1eb84-6629-47e6-8b46-a25c91d81a70 None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:36.010683 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca1eb84-6629-47e6-8b46-a25c91d81a70 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:36.011176 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca1eb84-6629-47e6-8b46-a25c91d81a70 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:36.013360 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca1eb84-6629-47e6-8b46-a25c91d81a70 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:36.031931 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 472/932] 10.222.0.61 () {56 vars in 1109 bytes} [Mon Aug 22 19:06:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:36.033097 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca1eb84-6629-47e6-8b46-a25c91d81a70 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:36.095371 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28927b2-05ae-4f87-9530-d33b4abee8e9 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:36.147291 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fa105a-ae18-4156-9f34-3a0e23aeaf49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:36.147995 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fa105a-ae18-4156-9f34-3a0e23aeaf49 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:36.148345 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fa105a-ae18-4156-9f34-3a0e23aeaf49 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:36.148688 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fa105a-ae18-4156-9f34-3a0e23aeaf49 None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:36.149647 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fa105a-ae18-4156-9f34-3a0e23aeaf49 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:36.150019 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fa105a-ae18-4156-9f34-3a0e23aeaf49 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:36.152094 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fa105a-ae18-4156-9f34-3a0e23aeaf49 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:36.162630 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 461/933] 10.222.0.61 () {56 vars in 1114 bytes} [Mon Aug 22 19:06:35 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:36.162630 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fa105a-ae18-4156-9f34-3a0e23aeaf49 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:36.184447 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca1eb84-6629-47e6-8b46-a25c91d81a70 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:36.212447 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:36.235329 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c846dd9b-49f1-4de1-bdf2-9548d4e5b62d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:36.236038 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c846dd9b-49f1-4de1-bdf2-9548d4e5b62d None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:36.236257 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c846dd9b-49f1-4de1-bdf2-9548d4e5b62d None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:36.236526 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c846dd9b-49f1-4de1-bdf2-9548d4e5b62d None admin] PATH_INFO: `/v3/services` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:36.237420 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c846dd9b-49f1-4de1-bdf2-9548d4e5b62d None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:36.237535 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c846dd9b-49f1-4de1-bdf2-9548d4e5b62d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:36.239082 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c846dd9b-49f1-4de1-bdf2-9548d4e5b62d None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:36.241367 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:36.250507 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 473/934] 10.222.0.61 () {56 vars in 1126 bytes} [Mon Aug 22 19:06:36 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:36.251391 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c846dd9b-49f1-4de1-bdf2-9548d4e5b62d None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:36.297835 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fa105a-ae18-4156-9f34-3a0e23aeaf49 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:36.342685 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f28a5e4-fda0-4fef-9c47-155b25da3532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:36.343674 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f28a5e4-fda0-4fef-9c47-155b25da3532 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:36.344591 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f28a5e4-fda0-4fef-9c47-155b25da3532 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:36.345089 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f28a5e4-fda0-4fef-9c47-155b25da3532 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:36.355034 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f28a5e4-fda0-4fef-9c47-155b25da3532 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:36.357241 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f28a5e4-fda0-4fef-9c47-155b25da3532 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:36.371177 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-3f28a5e4-fda0-4fef-9c47-155b25da3532 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 22 19:06:36.371177 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:36.371177 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:36.371177 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:36.371177 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:36.371177 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:36.371177 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:36.371177 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:36.371177 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:36.371177 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:36.371177 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:36.371177 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:36.371177 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 19:06:36.371177 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 19:06:36.371177 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 19:06:36.371177 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 19:06:36.372252 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:36.372252 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:36.372252 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 19:06:36.372252 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 19:06:36.372252 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 19:06:36.372252 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 19:06:36.372252 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:36.372252 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:36.372252 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:36.372252 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:36.372252 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:36.372252 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:36.372252 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:36.372252 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:36.372252 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:36.372252 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:36.372252 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:36.372252 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:36.373219 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:36.373219 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:36.373219 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 19:06:36.373219 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 19:06:36.373219 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 19:06:36.373219 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 19:06:36.373219 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 19:06:36.373219 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 19:06:36.373219 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 22 19:06:36.373219 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:06:36.373219 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 462/935] 10.222.0.61 () {56 vars in 1117 bytes} [Mon Aug 22 19:06:36 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:36.373219 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f28a5e4-fda0-4fef-9c47-155b25da3532 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:36.378304 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c846dd9b-49f1-4de1-bdf2-9548d4e5b62d None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:36.414711 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02593d84-9560-4a5c-8aa2-47e43c613260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:36.415314 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02593d84-9560-4a5c-8aa2-47e43c613260 None admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:36.415589 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02593d84-9560-4a5c-8aa2-47e43c613260 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:36.415886 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02593d84-9560-4a5c-8aa2-47e43c613260 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:36.416539 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02593d84-9560-4a5c-8aa2-47e43c613260 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:36.417891 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02593d84-9560-4a5c-8aa2-47e43c613260 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:36.438473 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 474/936] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 19:06:36 2022] POST /identity/v3/endpoints => generated 346 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:06:36.439286 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02593d84-9560-4a5c-8aa2-47e43c613260 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:36.498673 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:36.523492 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f28a5e4-fda0-4fef-9c47-155b25da3532 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:36.557940 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:36.573567 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02593d84-9560-4a5c-8aa2-47e43c613260 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:36.576125 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1f5c67-73a8-4bf3-9e83-ef8bd46e0c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:36.578604 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1f5c67-73a8-4bf3-9e83-ef8bd46e0c2b None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:36.579152 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1f5c67-73a8-4bf3-9e83-ef8bd46e0c2b None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:36.579609 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1f5c67-73a8-4bf3-9e83-ef8bd46e0c2b None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:36.580478 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1f5c67-73a8-4bf3-9e83-ef8bd46e0c2b None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:36.580972 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1f5c67-73a8-4bf3-9e83-ef8bd46e0c2b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:36.582623 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1f5c67-73a8-4bf3-9e83-ef8bd46e0c2b None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:36.591570 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 463/937] 10.222.0.61 () {56 vars in 1126 bytes} [Mon Aug 22 19:06:36 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:36.596301 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1f5c67-73a8-4bf3-9e83-ef8bd46e0c2b None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:36.614941 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05e4588-c558-4cac-8862-583408462373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:36.615482 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e4588-c558-4cac-8862-583408462373 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:36.615785 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e4588-c558-4cac-8862-583408462373 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:36.616051 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e4588-c558-4cac-8862-583408462373 None admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:36.616784 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05e4588-c558-4cac-8862-583408462373 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:36.617088 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05e4588-c558-4cac-8862-583408462373 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:36.618511 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05e4588-c558-4cac-8862-583408462373 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:36.643690 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 475/938] 10.222.0.61 () {56 vars in 1149 bytes} [Mon Aug 22 19:06:36 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:36.646018 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05e4588-c558-4cac-8862-583408462373 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:36.730118 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1f5c67-73a8-4bf3-9e83-ef8bd46e0c2b None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:36.752599 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:36.759265 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05e4588-c558-4cac-8862-583408462373 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:36.775162 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62af79dc-2315-480a-8aee-59565714c559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:36.776236 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62af79dc-2315-480a-8aee-59565714c559 None admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:36.776683 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62af79dc-2315-480a-8aee-59565714c559 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:36.777393 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62af79dc-2315-480a-8aee-59565714c559 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:36.778210 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62af79dc-2315-480a-8aee-59565714c559 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:36.779750 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62af79dc-2315-480a-8aee-59565714c559 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:36.796846 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68e9af3-93e1-4683-b5ea-06353dae0077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:36.797705 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68e9af3-93e1-4683-b5ea-06353dae0077 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:36.798080 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68e9af3-93e1-4683-b5ea-06353dae0077 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:36.798450 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68e9af3-93e1-4683-b5ea-06353dae0077 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:36.806288 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 464/939] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 19:06:36 2022] POST /identity/v3/endpoints => generated 347 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:06:36.812809 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68e9af3-93e1-4683-b5ea-06353dae0077 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:36.814659 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68e9af3-93e1-4683-b5ea-06353dae0077 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:36.832406 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-f68e9af3-93e1-4683-b5ea-06353dae0077 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:06:36.832406 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:36.832406 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:36.832406 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:36.832406 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:36.832406 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:36.832406 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:36.832406 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:36.832406 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:36.832406 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:36.832406 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:36.832406 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:36.832406 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 19:06:36.832406 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 19:06:36.832406 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 19:06:36.832406 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 19:06:36.833436 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:36.833436 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:36.833436 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:36.833436 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:36.833436 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:36.833436 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:36.833436 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:36.833436 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:36.833436 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:36.833436 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:36.833436 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:36.833436 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:36.833436 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:36.833436 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:36.833436 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:36.833436 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:36.833436 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 19:06:36.834331 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 19:06:36.834331 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:06:36.834331 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:06:36.834331 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 476/940] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:06:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 19:06:36.839971 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62af79dc-2315-480a-8aee-59565714c559 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:36.898874 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:36.977271 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62af79dc-2315-480a-8aee-59565714c559 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:37.013860 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838f0f61-fb04-472c-8926-4caaf0f3bdd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:37.015037 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838f0f61-fb04-472c-8926-4caaf0f3bdd6 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:37.015314 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838f0f61-fb04-472c-8926-4caaf0f3bdd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:37.015586 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838f0f61-fb04-472c-8926-4caaf0f3bdd6 None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:37.016309 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838f0f61-fb04-472c-8926-4caaf0f3bdd6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:37.016607 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838f0f61-fb04-472c-8926-4caaf0f3bdd6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:37.017967 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838f0f61-fb04-472c-8926-4caaf0f3bdd6 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:37.030341 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 465/941] 10.222.0.61 () {56 vars in 1109 bytes} [Mon Aug 22 19:06:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:37.032426 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338761f7-98f7-4298-9056-7127a5776982 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:37.032714 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338761f7-98f7-4298-9056-7127a5776982 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:37.032991 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338761f7-98f7-4298-9056-7127a5776982 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:37.034571 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 477/942] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:06:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:06:37.035739 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838f0f61-fb04-472c-8926-4caaf0f3bdd6 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:37.039624 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12292e80-b1dd-4dcd-8cb6-03748eab36ee None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:37.039935 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12292e80-b1dd-4dcd-8cb6-03748eab36ee None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:37.040214 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12292e80-b1dd-4dcd-8cb6-03748eab36ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:37.064536 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-12292e80-b1dd-4dcd-8cb6-03748eab36ee None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:06:37.065731 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-12292e80-b1dd-4dcd-8cb6-03748eab36ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:06:37.151541 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 478/943] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:06:37 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 19:06:37.156041 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:37.159605 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838f0f61-fb04-472c-8926-4caaf0f3bdd6 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:37.198326 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f94dd69-0e1f-47ae-b27c-dba28f99a165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:37.199429 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f94dd69-0e1f-47ae-b27c-dba28f99a165 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:37.199999 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f94dd69-0e1f-47ae-b27c-dba28f99a165 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:37.200510 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f94dd69-0e1f-47ae-b27c-dba28f99a165 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:37.210201 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f94dd69-0e1f-47ae-b27c-dba28f99a165 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:37.211673 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f94dd69-0e1f-47ae-b27c-dba28f99a165 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:37.224099 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-3f94dd69-0e1f-47ae-b27c-dba28f99a165 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 19:06:37.224099 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:37.224099 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:37.224099 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:37.224099 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:37.224099 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:37.224099 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:37.224099 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:37.224099 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:37.224099 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:37.224099 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:37.224099 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:37.224099 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 19:06:37.224099 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 19:06:37.224099 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 19:06:37.225058 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 19:06:37.225058 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:37.225058 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:37.225058 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:37.225058 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:37.225058 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:37.225058 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:37.225058 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:37.225058 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:37.225058 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:37.225058 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:37.225058 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:37.225058 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:37.225058 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:37.225058 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:37.225058 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:37.225058 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:37.226049 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 19:06:37.226049 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 19:06:37.226049 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 19:06:37.226049 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 19:06:37.226049 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 19:06:37.226049 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 19:06:37.226049 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 19:06:37.226049 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:06:37.226990 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 466/944] 10.222.0.61 () {56 vars in 1111 bytes} [Mon Aug 22 19:06:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:37.232012 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12292e80-b1dd-4dcd-8cb6-03748eab36ee None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:37.320867 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:37.344167 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12292e80-b1dd-4dcd-8cb6-03748eab36ee None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:37.380337 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7214b8-0e6f-48b1-906b-0579c68f7069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:37.380957 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7214b8-0e6f-48b1-906b-0579c68f7069 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:37.381238 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7214b8-0e6f-48b1-906b-0579c68f7069 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:37.381507 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7214b8-0e6f-48b1-906b-0579c68f7069 None admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:37.382210 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7214b8-0e6f-48b1-906b-0579c68f7069 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:37.382517 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7214b8-0e6f-48b1-906b-0579c68f7069 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:37.384035 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7214b8-0e6f-48b1-906b-0579c68f7069 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:37.396288 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 479/945] 10.222.0.61 () {56 vars in 1148 bytes} [Mon Aug 22 19:06:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:37.402025 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f94dd69-0e1f-47ae-b27c-dba28f99a165 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:37.420239 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7214b8-0e6f-48b1-906b-0579c68f7069 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:37.513886 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f94dd69-0e1f-47ae-b27c-dba28f99a165 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:37.543308 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7214b8-0e6f-48b1-906b-0579c68f7069 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:37.555590 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac11232-a826-42c3-8207-1d5623bc28ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:37.556625 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac11232-a826-42c3-8207-1d5623bc28ce None admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:37.557111 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac11232-a826-42c3-8207-1d5623bc28ce None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:37.557617 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac11232-a826-42c3-8207-1d5623bc28ce None admin] PATH_INFO: `/v3/projects/e07dc5ebdc504c6da706a2d21c0a3970/users/399396a8fe254da199bc48b0ca5243d0/roles/aa9eb04acf884c51ad2adb05380bc7a1` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:37.560153 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac11232-a826-42c3-8207-1d5623bc28ce None admin] RBAC: Authorizing `identity:create_grant(project_id=e07dc5ebdc504c6da706a2d21c0a3970, user_id=399396a8fe254da199bc48b0ca5243d0, role_id=aa9eb04acf884c51ad2adb05380bc7a1)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:37.562484 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac11232-a826-42c3-8207-1d5623bc28ce None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:37.583181 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 467/946] 10.222.0.61 () {58 vars in 1439 bytes} [Mon Aug 22 19:06:37 2022] PUT /identity/v3/projects/e07dc5ebdc504c6da706a2d21c0a3970/users/399396a8fe254da199bc48b0ca5243d0/roles/aa9eb04acf884c51ad2adb05380bc7a1 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:06:37.583625 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca74463-a47e-4444-91aa-a4d6c0cb18f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:37.584318 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca74463-a47e-4444-91aa-a4d6c0cb18f2 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:37.584608 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca74463-a47e-4444-91aa-a4d6c0cb18f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:37.584869 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca74463-a47e-4444-91aa-a4d6c0cb18f2 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:37.585623 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca74463-a47e-4444-91aa-a4d6c0cb18f2 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:37.587057 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca74463-a47e-4444-91aa-a4d6c0cb18f2 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:37.604789 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-cca74463-a47e-4444-91aa-a4d6c0cb18f2 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 22 19:06:37.604789 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:37.604789 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 19:06:37.604789 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 19:06:37.604789 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 19:06:37.604789 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 19:06:37.604789 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 19:06:37.604789 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 19:06:37.604789 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 22 19:06:37.604789 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:06:37.604789 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 19:06:37.604789 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:06:37.604789 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:37.604789 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:37.604789 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:37.604789 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:37.604789 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:37.605878 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:37.605878 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:37.605878 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:37.605878 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:37.605878 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:37.605878 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:37.605878 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 19:06:37.605878 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 19:06:37.605878 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 19:06:37.605878 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 19:06:37.605878 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:37.605878 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:37.605878 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:37.605878 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:37.605878 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:37.605878 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:37.607169 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:37.607169 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:37.607169 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:37.607169 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:37.607169 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:37.607169 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:37.607169 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:37.607169 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:37.607169 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:37.607169 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:37.607169 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 19:06:37.607169 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 19:06:37.607169 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 22 19:06:37.607169 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:06:37.607169 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 480/947] 10.222.0.61 () {56 vars in 1126 bytes} [Mon Aug 22 19:06:37 2022] GET /identity/v3/services/object-store => generated 93 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 19:06:37.612840 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac11232-a826-42c3-8207-1d5623bc28ce None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:37.638919 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:37.732504 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac11232-a826-42c3-8207-1d5623bc28ce None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:37.770863 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08117d97-be2e-47dd-ba2c-e5ac3bc5a542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:37.771740 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08117d97-be2e-47dd-ba2c-e5ac3bc5a542 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:37.772168 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08117d97-be2e-47dd-ba2c-e5ac3bc5a542 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:37.772613 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08117d97-be2e-47dd-ba2c-e5ac3bc5a542 None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:37.773393 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08117d97-be2e-47dd-ba2c-e5ac3bc5a542 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:37.773844 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08117d97-be2e-47dd-ba2c-e5ac3bc5a542 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:37.775491 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08117d97-be2e-47dd-ba2c-e5ac3bc5a542 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:37.781013 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 468/948] 10.222.0.61 () {56 vars in 1122 bytes} [Mon Aug 22 19:06:37 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 19:06:37.786204 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca74463-a47e-4444-91aa-a4d6c0cb18f2 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:37.789950 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:37.891944 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca74463-a47e-4444-91aa-a4d6c0cb18f2 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:37.926592 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fab6256-3e13-4923-b2c4-3e7b46a2c40a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:37.927208 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fab6256-3e13-4923-b2c4-3e7b46a2c40a None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:37.927490 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fab6256-3e13-4923-b2c4-3e7b46a2c40a None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:37.927799 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fab6256-3e13-4923-b2c4-3e7b46a2c40a None admin] PATH_INFO: `/v3/services` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:37.928439 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fab6256-3e13-4923-b2c4-3e7b46a2c40a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:37.928742 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fab6256-3e13-4923-b2c4-3e7b46a2c40a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:37.930117 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fab6256-3e13-4923-b2c4-3e7b46a2c40a None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:37.935633 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 481/949] 10.222.0.61 () {56 vars in 1122 bytes} [Mon Aug 22 19:06:37 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 19:06:37.941397 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08117d97-be2e-47dd-ba2c-e5ac3bc5a542 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:38.037813 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:38.055936 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08117d97-be2e-47dd-ba2c-e5ac3bc5a542 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:38.096414 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4700163e-d1e8-4aea-b8b8-5a7291a2a52d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:38.097018 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4700163e-d1e8-4aea-b8b8-5a7291a2a52d None admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:38.097299 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4700163e-d1e8-4aea-b8b8-5a7291a2a52d None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:38.097568 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4700163e-d1e8-4aea-b8b8-5a7291a2a52d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:38.098243 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4700163e-d1e8-4aea-b8b8-5a7291a2a52d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:38.099597 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4700163e-d1e8-4aea-b8b8-5a7291a2a52d None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:38.124811 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 469/950] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 19:06:37 2022] POST /identity/v3/endpoints => generated 349 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 19:06:38.578775 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196b53cb-1618-4d72-ad2f-9dcddc73dd3d None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:38.579940 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196b53cb-1618-4d72-ad2f-9dcddc73dd3d None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:38.580460 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196b53cb-1618-4d72-ad2f-9dcddc73dd3d None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:38.582319 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 482/951] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:06:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:06:38.587938 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac10522b-942f-440b-9fcf-011f322dae48 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:38.588813 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac10522b-942f-440b-9fcf-011f322dae48 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:38.589309 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac10522b-942f-440b-9fcf-011f322dae48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:38.612009 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-ac10522b-942f-440b-9fcf-011f322dae48 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:06:38.613574 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-ac10522b-942f-440b-9fcf-011f322dae48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:06:38.691862 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 470/952] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:06:38 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 19:06:38.923410 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7b6046-a0cf-47f9-9ff7-d65318798d36 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:38.923637 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7b6046-a0cf-47f9-9ff7-d65318798d36 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:38.923935 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7b6046-a0cf-47f9-9ff7-d65318798d36 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:38.925583 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 483/953] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:06:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:06:38.930767 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f5d0d3-a9ec-46dd-b1fd-d2cdb9b7645d None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:38.931052 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f5d0d3-a9ec-46dd-b1fd-d2cdb9b7645d None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:38.931324 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f5d0d3-a9ec-46dd-b1fd-d2cdb9b7645d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:38.949235 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-b6f5d0d3-a9ec-46dd-b1fd-d2cdb9b7645d None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:06:38.950425 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-b6f5d0d3-a9ec-46dd-b1fd-d2cdb9b7645d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:06:38.968682 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7b6046-a0cf-47f9-9ff7-d65318798d36 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:39.013696 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:39.031855 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 471/954] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:06:38 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 19:06:39.046265 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:39.084575 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7b6046-a0cf-47f9-9ff7-d65318798d36 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:39.121645 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76327dfe-a99c-4d67-a584-7bc8b72ad4e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:39.122328 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76327dfe-a99c-4d67-a584-7bc8b72ad4e7 None admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:39.122609 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76327dfe-a99c-4d67-a584-7bc8b72ad4e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:39.122884 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76327dfe-a99c-4d67-a584-7bc8b72ad4e7 None admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:39.123531 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76327dfe-a99c-4d67-a584-7bc8b72ad4e7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:39.125107 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76327dfe-a99c-4d67-a584-7bc8b72ad4e7 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:39.127819 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: WARNING keystone.server.flask.common [None req-76327dfe-a99c-4d67-a584-7bc8b72ad4e7 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 22 19:06:39.149178 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.sql.core [None req-76327dfe-a99c-4d67-a584-7bc8b72ad4e7 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 22 19:06:39.149178 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 22 19:06:39.149178 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [parameters: {'id': 'c69ce5babcaa43a687736dc8bb94b7b9', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 22 19:06:39.149178 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81533) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 22 19:06:39.155882 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-76327dfe-a99c-4d67-a584-7bc8b72ad4e7 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c69ce5babcaa43a687736dc8bb94b7b9.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c69ce5babcaa43a687736dc8bb94b7b9. Aug 22 19:06:39.155882 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:39.155882 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 22 19:06:39.155882 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 22 19:06:39.155882 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 22 19:06:39.155882 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 22 19:06:39.155882 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 22 19:06:39.155882 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application result = self._query(query) Aug 22 19:06:39.155882 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 22 19:06:39.155882 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application conn.query(q) Aug 22 19:06:39.155882 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 22 19:06:39.155882 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 22 19:06:39.155882 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 22 19:06:39.156686 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application result.read() Aug 22 19:06:39.156686 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 22 19:06:39.156686 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 22 19:06:39.156686 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 22 19:06:39.156686 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 22 19:06:39.156686 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 22 19:06:39.156686 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 22 19:06:39.156686 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 22 19:06:39.156686 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 22 19:06:39.156686 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 22 19:06:39.156686 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:06:39.156686 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 22 19:06:39.156686 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:06:39.156686 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:39.156686 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 22 19:06:39.156686 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 22 19:06:39.156686 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 22 19:06:39.156686 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 22 19:06:39.158060 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 22 19:06:39.158060 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application next(self.gen) Aug 22 19:06:39.158060 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 22 19:06:39.158060 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application yield resource Aug 22 19:06:39.158060 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 22 19:06:39.158060 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application next(self.gen) Aug 22 19:06:39.158060 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 22 19:06:39.158060 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self.session.rollback() Aug 22 19:06:39.158060 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 22 19:06:39.158060 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self.force_reraise() Aug 22 19:06:39.158060 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 22 19:06:39.158060 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise self.value Aug 22 19:06:39.158060 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 22 19:06:39.158060 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 22 19:06:39.158060 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 22 19:06:39.158060 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application session.commit() Aug 22 19:06:39.158060 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 22 19:06:39.158961 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 22 19:06:39.158961 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 22 19:06:39.158961 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self._prepare_impl() Aug 22 19:06:39.158961 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 22 19:06:39.158961 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self.session.flush() Aug 22 19:06:39.158961 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 22 19:06:39.158961 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self._flush(objects) Aug 22 19:06:39.158961 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 22 19:06:39.158961 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 22 19:06:39.158961 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 22 19:06:39.158961 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application compat.raise_( Aug 22 19:06:39.158961 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 22 19:06:39.158961 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception Aug 22 19:06:39.158961 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 22 19:06:39.158961 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application flush_context.execute() Aug 22 19:06:39.158961 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 22 19:06:39.158961 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rec.execute(self) Aug 22 19:06:39.159926 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 22 19:06:39.159926 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 22 19:06:39.159926 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 22 19:06:39.159926 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 22 19:06:39.159926 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 22 19:06:39.159926 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 22 19:06:39.159926 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 22 19:06:39.159926 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 22 19:06:39.159926 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 22 19:06:39.159926 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 22 19:06:39.159926 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 22 19:06:39.159926 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 22 19:06:39.159926 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 22 19:06:39.159926 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 22 19:06:39.159926 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 22 19:06:39.160711 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 22 19:06:39.160711 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 22 19:06:39.160711 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception Aug 22 19:06:39.160711 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 22 19:06:39.160711 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 22 19:06:39.160711 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 22 19:06:39.160711 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 22 19:06:39.160711 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 22 19:06:39.160711 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application result = self._query(query) Aug 22 19:06:39.160711 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 22 19:06:39.160711 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application conn.query(q) Aug 22 19:06:39.160711 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 22 19:06:39.160711 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 22 19:06:39.160711 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 22 19:06:39.160711 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application result.read() Aug 22 19:06:39.160711 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 22 19:06:39.160711 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 22 19:06:39.161617 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 22 19:06:39.161617 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 22 19:06:39.161617 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 22 19:06:39.161617 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 22 19:06:39.161617 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 22 19:06:39.161617 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 22 19:06:39.161617 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 22 19:06:39.161617 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 22 19:06:39.161617 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [parameters: {'id': 'c69ce5babcaa43a687736dc8bb94b7b9', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 22 19:06:39.161617 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 22 19:06:39.161617 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:06:39.161617 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 19:06:39.161617 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:06:39.161617 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:39.161617 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 22 19:06:39.162472 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 22 19:06:39.162472 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 22 19:06:39.162472 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 22 19:06:39.162472 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 22 19:06:39.162472 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:06:39.162472 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 19:06:39.162472 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:06:39.162472 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:39.162472 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:39.162472 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:39.162472 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:39.162472 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:39.162472 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:39.162472 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:39.162472 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:39.162472 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:39.162472 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:39.163447 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:39.163447 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 22 19:06:39.163447 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 22 19:06:39.163447 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:39.163447 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:39.163447 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 22 19:06:39.163447 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 22 19:06:39.163447 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c69ce5babcaa43a687736dc8bb94b7b9. Aug 22 19:06:39.163447 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:06:39.163447 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 484/955] 10.222.0.61 () {60 vars in 1140 bytes} [Mon Aug 22 19:06:38 2022] POST /identity/v3/projects => generated 262 bytes in 190 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 22 19:06:39.171415 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f5d0d3-a9ec-46dd-b1fd-d2cdb9b7645d None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:39.280632 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f5d0d3-a9ec-46dd-b1fd-d2cdb9b7645d None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:39.316197 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda03822-803a-4f72-a04c-a90323204d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:39.316832 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda03822-803a-4f72-a04c-a90323204d1d None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:39.317108 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda03822-803a-4f72-a04c-a90323204d1d None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:39.317380 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda03822-803a-4f72-a04c-a90323204d1d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:39.324290 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda03822-803a-4f72-a04c-a90323204d1d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:39.325556 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76327dfe-a99c-4d67-a584-7bc8b72ad4e7 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:39.328435 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda03822-803a-4f72-a04c-a90323204d1d None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:39.344196 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-bda03822-803a-4f72-a04c-a90323204d1d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 19:06:39.344196 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:39.344196 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:39.344196 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:39.344196 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:39.344196 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:39.344196 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:39.344196 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:39.344196 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:39.344196 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:39.344196 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:39.344196 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:39.344196 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 19:06:39.344196 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 19:06:39.344196 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 19:06:39.345372 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 19:06:39.345372 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:39.345372 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:39.345372 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:39.345372 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:39.345372 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:39.345372 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:39.345372 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:39.345372 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:39.345372 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:39.345372 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:39.345372 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:39.345372 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:39.345372 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:39.345372 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:39.345372 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:39.345372 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:39.348757 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 19:06:39.348757 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 19:06:39.348757 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 19:06:39.348757 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 19:06:39.348757 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 19:06:39.348757 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 19:06:39.348757 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 19:06:39.348757 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:06:39.348757 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 472/956] 10.222.0.61 () {56 vars in 1111 bytes} [Mon Aug 22 19:06:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:39.351543 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda03822-803a-4f72-a04c-a90323204d1d None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:39.423875 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:39.458610 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76327dfe-a99c-4d67-a584-7bc8b72ad4e7 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:39.485873 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda03822-803a-4f72-a04c-a90323204d1d None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:39.486848 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:39.502604 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8cc54c-f3e8-4325-b9f8-5ed88e5d1c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:39.503384 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8cc54c-f3e8-4325-b9f8-5ed88e5d1c89 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:39.503797 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8cc54c-f3e8-4325-b9f8-5ed88e5d1c89 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:39.504168 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8cc54c-f3e8-4325-b9f8-5ed88e5d1c89 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:39.504854 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8cc54c-f3e8-4325-b9f8-5ed88e5d1c89 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:39.506719 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8cc54c-f3e8-4325-b9f8-5ed88e5d1c89 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:39.517988 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-bc8cc54c-f3e8-4325-b9f8-5ed88e5d1c89 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 22 19:06:39.517988 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:39.517988 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 19:06:39.517988 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 19:06:39.517988 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 19:06:39.517988 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 19:06:39.517988 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 19:06:39.517988 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 19:06:39.517988 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 22 19:06:39.517988 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:06:39.517988 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 19:06:39.517988 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:06:39.517988 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:39.517988 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:39.517988 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:39.517988 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:39.517988 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:39.519328 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:39.519328 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:39.519328 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:39.519328 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:39.519328 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:39.519328 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:39.519328 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 19:06:39.519328 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 19:06:39.519328 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 19:06:39.519328 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 19:06:39.519328 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:39.519328 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:39.519328 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:39.519328 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:39.519328 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:39.519328 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:39.520508 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:39.520508 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:39.520508 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:39.520508 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:39.520508 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:39.520508 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:39.520508 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:39.520508 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:39.520508 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:39.520508 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:39.520508 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 19:06:39.520508 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 19:06:39.520508 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 22 19:06:39.520508 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:06:39.523321 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 485/957] 10.222.0.61 () {56 vars in 1111 bytes} [Mon Aug 22 19:06:39 2022] GET /identity/v3/services/compute => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 19:06:39.529483 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8cc54c-f3e8-4325-b9f8-5ed88e5d1c89 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:39.549577 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcf1a06-c3af-4bbb-8ea8-7c2e09472c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:39.550606 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcf1a06-c3af-4bbb-8ea8-7c2e09472c5f None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:39.551451 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcf1a06-c3af-4bbb-8ea8-7c2e09472c5f None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:39.551921 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcf1a06-c3af-4bbb-8ea8-7c2e09472c5f None admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:39.552744 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcf1a06-c3af-4bbb-8ea8-7c2e09472c5f None admin] RBAC: Adding query filter params (name=service) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:39.553199 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcf1a06-c3af-4bbb-8ea8-7c2e09472c5f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:39.555286 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcf1a06-c3af-4bbb-8ea8-7c2e09472c5f None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:39.587672 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 473/958] 10.222.0.61 () {56 vars in 1112 bytes} [Mon Aug 22 19:06:39 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:39.599247 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb74024d-c41b-455c-b868-cc109353ce8d None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:39.599927 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb74024d-c41b-455c-b868-cc109353ce8d None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:39.599927 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb74024d-c41b-455c-b868-cc109353ce8d None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:39.601488 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 474/959] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:06:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 19:06:39.607092 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fec4b00-8eec-46b0-9d9b-0b9f087e1f8f None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:39.607381 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fec4b00-8eec-46b0-9d9b-0b9f087e1f8f None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:39.607727 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fec4b00-8eec-46b0-9d9b-0b9f087e1f8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:39.635505 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-0fec4b00-8eec-46b0-9d9b-0b9f087e1f8f None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:06:39.637464 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-0fec4b00-8eec-46b0-9d9b-0b9f087e1f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:06:39.652317 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8cc54c-f3e8-4325-b9f8-5ed88e5d1c89 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:39.693240 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c849c508-cccf-49d4-a3e1-c275154c986e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:39.694060 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c849c508-cccf-49d4-a3e1-c275154c986e None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:39.694602 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c849c508-cccf-49d4-a3e1-c275154c986e None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:39.694975 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c849c508-cccf-49d4-a3e1-c275154c986e None admin] PATH_INFO: `/v3/services` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:39.695812 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c849c508-cccf-49d4-a3e1-c275154c986e None admin] RBAC: Adding query filter params (name=compute) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:39.696223 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c849c508-cccf-49d4-a3e1-c275154c986e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:39.697719 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c849c508-cccf-49d4-a3e1-c275154c986e None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:39.707232 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 486/960] 10.222.0.61 () {56 vars in 1112 bytes} [Mon Aug 22 19:06:39 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:39.712922 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c849c508-cccf-49d4-a3e1-c275154c986e None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:39.735965 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 475/961] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:06:39 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 22 19:06:39.774362 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:39.830721 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c849c508-cccf-49d4-a3e1-c275154c986e None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:39.866722 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541e4088-af8b-476c-80a9-31cadcd16acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:39.867567 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541e4088-af8b-476c-80a9-31cadcd16acb None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:39.868025 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541e4088-af8b-476c-80a9-31cadcd16acb None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:39.868460 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541e4088-af8b-476c-80a9-31cadcd16acb None admin] PATH_INFO: `/v3/services` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:39.869233 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541e4088-af8b-476c-80a9-31cadcd16acb None admin] RBAC: Adding query filter params (type=compute) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:39.869689 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541e4088-af8b-476c-80a9-31cadcd16acb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:39.871198 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541e4088-af8b-476c-80a9-31cadcd16acb None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:39.879559 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 487/962] 10.222.0.61 () {56 vars in 1112 bytes} [Mon Aug 22 19:06:39 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 19:06:40.003015 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fec4b00-8eec-46b0-9d9b-0b9f087e1f8f None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:40.053136 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:40.055312 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caf7d7d-2915-4886-8e02-c27a01ac4cef None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:40.055598 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caf7d7d-2915-4886-8e02-c27a01ac4cef None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:40.055915 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caf7d7d-2915-4886-8e02-c27a01ac4cef None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:40.057515 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 488/963] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:06:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:06:40.064287 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501996f3-57a2-4bfb-86cc-dfbe5cceb39f None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:40.064567 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501996f3-57a2-4bfb-86cc-dfbe5cceb39f None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:40.064839 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501996f3-57a2-4bfb-86cc-dfbe5cceb39f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:40.084855 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-501996f3-57a2-4bfb-86cc-dfbe5cceb39f None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:06:40.086292 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-501996f3-57a2-4bfb-86cc-dfbe5cceb39f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:06:40.112937 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fec4b00-8eec-46b0-9d9b-0b9f087e1f8f None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:40.158973 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4f901d-6bdc-40df-a216-a77f23fa1660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:40.160355 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4f901d-6bdc-40df-a216-a77f23fa1660 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:40.160891 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4f901d-6bdc-40df-a216-a77f23fa1660 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:40.161626 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4f901d-6bdc-40df-a216-a77f23fa1660 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:40.175732 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 489/964] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:06:40 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 19:06:40.180635 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4f901d-6bdc-40df-a216-a77f23fa1660 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 19:06:40.185765 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-4b4f901d-6bdc-40df-a216-a77f23fa1660 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 22 19:06:40.185765 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:40.185765 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:40.185765 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:40.185765 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:40.185765 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:40.185765 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:40.185765 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:40.185765 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:40.185765 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:40.185765 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:40.185765 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:40.185765 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 19:06:40.185765 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 19:06:40.185765 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 19:06:40.185765 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 19:06:40.186788 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 19:06:40.186788 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 19:06:40.186788 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 19:06:40.186788 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 19:06:40.186788 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:40.186788 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:40.186788 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:40.186788 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:40.186788 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:40.186788 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:40.186788 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:40.186788 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:40.186788 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:40.186788 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:40.186788 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:40.186788 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:40.188062 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:40.188062 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:40.188062 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:40.188062 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:40.188062 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 19:06:40.188062 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 19:06:40.188062 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 19:06:40.188062 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 19:06:40.188062 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 19:06:40.188062 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 19:06:40.188062 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 22 19:06:40.188062 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:06:40.190475 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 476/965] 10.222.0.61 () {56 vars in 1120 bytes} [Mon Aug 22 19:06:40 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:40.196648 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501996f3-57a2-4bfb-86cc-dfbe5cceb39f None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:40.247665 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:40.310943 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501996f3-57a2-4bfb-86cc-dfbe5cceb39f None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:40.348141 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d71a649-3566-4d2f-96c6-fcae43a834d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:40.349109 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d71a649-3566-4d2f-96c6-fcae43a834d5 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:40.349624 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d71a649-3566-4d2f-96c6-fcae43a834d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:40.350105 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d71a649-3566-4d2f-96c6-fcae43a834d5 None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:40.351021 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d71a649-3566-4d2f-96c6-fcae43a834d5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:40.351547 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d71a649-3566-4d2f-96c6-fcae43a834d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:40.353171 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d71a649-3566-4d2f-96c6-fcae43a834d5 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:40.362575 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 490/966] 10.222.0.61 () {56 vars in 1115 bytes} [Mon Aug 22 19:06:40 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 19:06:40.368653 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4f901d-6bdc-40df-a216-a77f23fa1660 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:40.440155 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d71a649-3566-4d2f-96c6-fcae43a834d5 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:40.474788 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4f901d-6bdc-40df-a216-a77f23fa1660 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:40.482150 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:40.512605 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da23f231-d5c3-4f13-84bc-b8b74d7bad52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:40.513255 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da23f231-d5c3-4f13-84bc-b8b74d7bad52 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:40.513899 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da23f231-d5c3-4f13-84bc-b8b74d7bad52 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:40.513899 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da23f231-d5c3-4f13-84bc-b8b74d7bad52 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:40.525191 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da23f231-d5c3-4f13-84bc-b8b74d7bad52 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:40.526633 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da23f231-d5c3-4f13-84bc-b8b74d7bad52 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:40.541620 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-da23f231-d5c3-4f13-84bc-b8b74d7bad52 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:06:40.541620 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:40.541620 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:40.541620 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:40.541620 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:40.541620 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:40.541620 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:40.541620 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:40.541620 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:40.541620 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:40.541620 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:40.541620 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:40.541620 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 19:06:40.541620 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 19:06:40.541620 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 19:06:40.541620 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 19:06:40.542670 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:40.542670 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:40.542670 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:40.542670 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:40.542670 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:40.542670 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:40.542670 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:40.542670 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:40.542670 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:40.542670 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:40.542670 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:40.542670 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:40.542670 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:40.542670 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:40.542670 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:40.542670 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:40.542670 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 19:06:40.543622 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 19:06:40.543622 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:06:40.543622 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:06:40.548048 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 477/967] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:06:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:40.549309 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da23f231-d5c3-4f13-84bc-b8b74d7bad52 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:40.554176 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d71a649-3566-4d2f-96c6-fcae43a834d5 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:40.568025 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:40.592799 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2fb1c0-2487-4209-bf6d-338e21ea910c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:40.593822 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2fb1c0-2487-4209-bf6d-338e21ea910c None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:40.594345 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2fb1c0-2487-4209-bf6d-338e21ea910c None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:40.594847 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2fb1c0-2487-4209-bf6d-338e21ea910c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:40.595706 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2fb1c0-2487-4209-bf6d-338e21ea910c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:40.597289 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2fb1c0-2487-4209-bf6d-338e21ea910c None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:40.606736 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-5a2fb1c0-2487-4209-bf6d-338e21ea910c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 22 19:06:40.606736 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:40.606736 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 19:06:40.606736 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 19:06:40.606736 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 19:06:40.606736 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 19:06:40.606736 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 19:06:40.606736 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 19:06:40.606736 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 22 19:06:40.606736 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:06:40.606736 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 19:06:40.606736 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:06:40.606736 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:40.606736 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:40.606736 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:40.606736 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:40.606736 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:40.607797 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:40.607797 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:40.607797 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:40.607797 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:40.607797 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:40.607797 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:40.607797 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 19:06:40.607797 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 19:06:40.607797 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 19:06:40.607797 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 19:06:40.607797 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:40.607797 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:40.607797 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:40.607797 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:40.607797 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:40.607797 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:40.609193 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:40.609193 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:40.609193 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:40.609193 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:40.609193 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:40.609193 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:40.609193 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:40.609193 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:40.609193 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:40.609193 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:40.609193 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 19:06:40.609193 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 19:06:40.609193 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 22 19:06:40.609193 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:06:40.613039 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 491/968] 10.222.0.61 () {56 vars in 1126 bytes} [Mon Aug 22 19:06:40 2022] GET /identity/v3/services/object-store => generated 93 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:40.618671 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2fb1c0-2487-4209-bf6d-338e21ea910c None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:40.668308 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da23f231-d5c3-4f13-84bc-b8b74d7bad52 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:40.715781 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e21d9aa-2638-483a-8807-9d9b4ee2b301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:40.716485 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e21d9aa-2638-483a-8807-9d9b4ee2b301 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:40.716766 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e21d9aa-2638-483a-8807-9d9b4ee2b301 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:40.717043 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e21d9aa-2638-483a-8807-9d9b4ee2b301 None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:40.718600 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e21d9aa-2638-483a-8807-9d9b4ee2b301 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:40.718908 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e21d9aa-2638-483a-8807-9d9b4ee2b301 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:40.720651 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e21d9aa-2638-483a-8807-9d9b4ee2b301 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:40.741841 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 478/969] 10.222.0.61 () {56 vars in 1109 bytes} [Mon Aug 22 19:06:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 19:06:40.743151 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e21d9aa-2638-483a-8807-9d9b4ee2b301 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:40.749201 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2fb1c0-2487-4209-bf6d-338e21ea910c None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:40.768929 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:40.785984 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883a9073-5595-49f3-b417-cbde068d78a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:40.786703 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883a9073-5595-49f3-b417-cbde068d78a0 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:40.786991 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883a9073-5595-49f3-b417-cbde068d78a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:40.787264 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883a9073-5595-49f3-b417-cbde068d78a0 None admin] PATH_INFO: `/v3/services` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:40.788106 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883a9073-5595-49f3-b417-cbde068d78a0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:40.788413 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883a9073-5595-49f3-b417-cbde068d78a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:40.789795 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883a9073-5595-49f3-b417-cbde068d78a0 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:40.800582 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 492/970] 10.222.0.61 () {56 vars in 1122 bytes} [Mon Aug 22 19:06:40 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:40.802427 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883a9073-5595-49f3-b417-cbde068d78a0 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:40.851422 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:40.869122 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e21d9aa-2638-483a-8807-9d9b4ee2b301 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:40.907868 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cf54b1-d57a-4171-92ed-031481612f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:40.908857 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cf54b1-d57a-4171-92ed-031481612f9a None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:40.909344 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cf54b1-d57a-4171-92ed-031481612f9a None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:40.909845 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cf54b1-d57a-4171-92ed-031481612f9a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:40.917600 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cf54b1-d57a-4171-92ed-031481612f9a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:40.919295 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cf54b1-d57a-4171-92ed-031481612f9a None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:40.932208 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-22cf54b1-d57a-4171-92ed-031481612f9a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 22 19:06:40.932208 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:40.932208 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:40.932208 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:40.932208 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:40.932208 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:40.932208 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:40.932208 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:40.932208 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:40.932208 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:40.932208 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:40.932208 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:40.932208 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 19:06:40.932208 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 19:06:40.932208 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 19:06:40.932208 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 19:06:40.933285 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:40.933285 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:40.933285 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 19:06:40.933285 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 19:06:40.933285 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 19:06:40.933285 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 19:06:40.933285 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:40.933285 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:40.933285 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:40.933285 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:40.933285 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:40.933285 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:40.933285 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:40.933285 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:40.933285 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:40.933285 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:40.933285 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:40.933285 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:40.934265 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:40.934265 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:40.934265 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 19:06:40.934265 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 19:06:40.934265 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 19:06:40.934265 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 19:06:40.934265 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 19:06:40.934265 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 19:06:40.934265 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 22 19:06:40.934265 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:06:40.941599 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883a9073-5595-49f3-b417-cbde068d78a0 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:40.941870 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 479/971] 10.222.0.61 () {56 vars in 1117 bytes} [Mon Aug 22 19:06:40 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:40.942891 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cf54b1-d57a-4171-92ed-031481612f9a None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:40.984935 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787698b6-9644-4a36-9afa-7b2a9886ae25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:40.985967 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787698b6-9644-4a36-9afa-7b2a9886ae25 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:40.986473 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787698b6-9644-4a36-9afa-7b2a9886ae25 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:40.986954 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787698b6-9644-4a36-9afa-7b2a9886ae25 None admin] PATH_INFO: `/v3/services` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:40.988034 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787698b6-9644-4a36-9afa-7b2a9886ae25 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:40.988588 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787698b6-9644-4a36-9afa-7b2a9886ae25 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:40.990469 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787698b6-9644-4a36-9afa-7b2a9886ae25 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:41.001418 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 493/972] 10.222.0.61 () {56 vars in 1122 bytes} [Mon Aug 22 19:06:40 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:41.002663 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787698b6-9644-4a36-9afa-7b2a9886ae25 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:41.036898 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:41.061632 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cf54b1-d57a-4171-92ed-031481612f9a None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:41.098954 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f019412-735d-42c4-9039-ecdad164519a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:41.099746 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f019412-735d-42c4-9039-ecdad164519a None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:41.100129 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f019412-735d-42c4-9039-ecdad164519a None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:41.100515 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f019412-735d-42c4-9039-ecdad164519a None admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:41.101354 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f019412-735d-42c4-9039-ecdad164519a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:41.101767 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f019412-735d-42c4-9039-ecdad164519a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:41.103339 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f019412-735d-42c4-9039-ecdad164519a None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:41.124151 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:41.127006 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 480/973] 10.222.0.61 () {56 vars in 1149 bytes} [Mon Aug 22 19:06:40 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:41.128190 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f019412-735d-42c4-9039-ecdad164519a None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:41.129216 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787698b6-9644-4a36-9afa-7b2a9886ae25 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:41.178070 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161be5fa-9949-4d9d-aaea-671279091358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:41.179497 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161be5fa-9949-4d9d-aaea-671279091358 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:41.180055 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161be5fa-9949-4d9d-aaea-671279091358 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:41.180484 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161be5fa-9949-4d9d-aaea-671279091358 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:41.181610 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161be5fa-9949-4d9d-aaea-671279091358 None admin] RBAC: Adding query filter params (interface=admin, service_id=4983e00fc60c4c0fafb5e2395ad5d267, region_id=RegionOne) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:41.182098 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161be5fa-9949-4d9d-aaea-671279091358 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:41.184392 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161be5fa-9949-4d9d-aaea-671279091358 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:41.191730 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 494/974] 10.222.0.61 () {56 vars in 1249 bytes} [Mon Aug 22 19:06:41 2022] GET /identity/v3/endpoints?service_id=4983e00fc60c4c0fafb5e2395ad5d267&interface=admin®ion_id=RegionOne => generated 193 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:41.197115 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161be5fa-9949-4d9d-aaea-671279091358 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:41.253251 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f019412-735d-42c4-9039-ecdad164519a None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:41.307504 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa35733-0039-4b49-a138-c93009ef349d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:41.308171 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa35733-0039-4b49-a138-c93009ef349d None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:41.308443 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa35733-0039-4b49-a138-c93009ef349d None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:41.308715 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa35733-0039-4b49-a138-c93009ef349d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:41.313206 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:41.324130 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161be5fa-9949-4d9d-aaea-671279091358 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:41.372821 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713497ea-e916-40b8-9210-8070a34605ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:41.373438 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713497ea-e916-40b8-9210-8070a34605ff None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:41.373754 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713497ea-e916-40b8-9210-8070a34605ff None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:41.374031 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713497ea-e916-40b8-9210-8070a34605ff None admin] PATH_INFO: `/v3/services` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:41.374705 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713497ea-e916-40b8-9210-8070a34605ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:41.376993 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713497ea-e916-40b8-9210-8070a34605ff None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:41.387676 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 495/975] 10.222.0.61 () {56 vars in 1087 bytes} [Mon Aug 22 19:06:41 2022] GET /identity/v3/services => generated 1497 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:06:41.399921 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa35733-0039-4b49-a138-c93009ef349d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:41.401543 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa35733-0039-4b49-a138-c93009ef349d None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:41.519389 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 481/976] 10.222.0.61 () {58 vars in 1282 bytes} [Mon Aug 22 19:06:41 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 22 19:06:41.526405 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713497ea-e916-40b8-9210-8070a34605ff None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:41.554932 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:41.633504 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713497ea-e916-40b8-9210-8070a34605ff None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:41.639153 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fd4ea2-1ed3-4814-b6fe-7313053361f5 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:41.640022 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fd4ea2-1ed3-4814-b6fe-7313053361f5 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:41.640299 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fd4ea2-1ed3-4814-b6fe-7313053361f5 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:41.641911 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 482/977] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:06:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 19:06:41.646884 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42131cac-36d6-4479-8d33-1e82d66cf8d7 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:41.647166 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42131cac-36d6-4479-8d33-1e82d66cf8d7 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:41.647445 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42131cac-36d6-4479-8d33-1e82d66cf8d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:41.669323 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-42131cac-36d6-4479-8d33-1e82d66cf8d7 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:06:41.670578 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-42131cac-36d6-4479-8d33-1e82d66cf8d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:06:41.671828 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb86e9e-6a4c-4d47-a94a-726f65e11dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:41.672470 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb86e9e-6a4c-4d47-a94a-726f65e11dad None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:41.672746 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb86e9e-6a4c-4d47-a94a-726f65e11dad None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:41.673296 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb86e9e-6a4c-4d47-a94a-726f65e11dad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:41.687768 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb86e9e-6a4c-4d47-a94a-726f65e11dad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:41.689978 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb86e9e-6a4c-4d47-a94a-726f65e11dad None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:41.701878 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:41.709399 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-8bb86e9e-6a4c-4d47-a94a-726f65e11dad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:06:41.709399 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:41.709399 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:41.709399 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:41.709399 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:41.709399 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:41.709399 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:41.709399 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:41.709399 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:41.709399 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:41.709399 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:41.709399 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:41.709399 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 19:06:41.709399 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 19:06:41.709399 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 19:06:41.709399 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 19:06:41.710574 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:41.710574 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:41.710574 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:41.710574 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:41.710574 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:41.710574 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:41.710574 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:41.710574 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:41.710574 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:41.710574 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:41.710574 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:41.710574 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:41.710574 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:41.710574 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:41.710574 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:41.710574 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:41.710574 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 19:06:41.711751 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 19:06:41.711751 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:06:41.711751 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:06:41.711751 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 496/978] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:06:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:41.716570 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb86e9e-6a4c-4d47-a94a-726f65e11dad None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:41.759766 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 483/979] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:06:41 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 22 19:06:41.812027 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:41.831066 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609fc26f-c908-41da-a83a-05a0de4f3785 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:41.831347 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609fc26f-c908-41da-a83a-05a0de4f3785 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:41.831621 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609fc26f-c908-41da-a83a-05a0de4f3785 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:41.832465 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb86e9e-6a4c-4d47-a94a-726f65e11dad None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:41.834435 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 484/980] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:06:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 19:06:41.838538 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def2f12f-ef92-4539-92f4-9c410a94205d None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:41.838819 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def2f12f-ef92-4539-92f4-9c410a94205d None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:41.839094 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def2f12f-ef92-4539-92f4-9c410a94205d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:41.859491 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-def2f12f-ef92-4539-92f4-9c410a94205d None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:06:41.860904 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-def2f12f-ef92-4539-92f4-9c410a94205d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:06:41.875431 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98de7a22-ad04-44b0-82b8-f50c52e25f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:41.876805 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98de7a22-ad04-44b0-82b8-f50c52e25f3c None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:41.877243 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98de7a22-ad04-44b0-82b8-f50c52e25f3c None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:41.877918 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98de7a22-ad04-44b0-82b8-f50c52e25f3c None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:41.878974 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98de7a22-ad04-44b0-82b8-f50c52e25f3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:41.879453 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98de7a22-ad04-44b0-82b8-f50c52e25f3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:41.881383 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98de7a22-ad04-44b0-82b8-f50c52e25f3c None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:41.901514 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 497/981] 10.222.0.61 () {56 vars in 1109 bytes} [Mon Aug 22 19:06:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:41.906232 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98de7a22-ad04-44b0-82b8-f50c52e25f3c None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:41.943764 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 485/982] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:06:41 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 19:06:42.009636 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98de7a22-ad04-44b0-82b8-f50c52e25f3c None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:42.034932 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def2f12f-ef92-4539-92f4-9c410a94205d None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:42.047183 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9534d3bf-046b-4bf1-b112-03c3cae23c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:42.047860 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9534d3bf-046b-4bf1-b112-03c3cae23c88 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:42.048134 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9534d3bf-046b-4bf1-b112-03c3cae23c88 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:42.048400 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9534d3bf-046b-4bf1-b112-03c3cae23c88 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:42.057483 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9534d3bf-046b-4bf1-b112-03c3cae23c88 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:42.058913 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9534d3bf-046b-4bf1-b112-03c3cae23c88 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:42.061167 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:42.074464 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-9534d3bf-046b-4bf1-b112-03c3cae23c88 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 19:06:42.074464 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:42.074464 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:42.074464 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:42.074464 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:42.074464 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:42.074464 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:42.074464 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:42.074464 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:42.074464 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:42.074464 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:42.074464 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:42.074464 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 19:06:42.074464 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 19:06:42.074464 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 19:06:42.075385 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 19:06:42.075385 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:42.075385 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:42.075385 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:42.075385 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:42.075385 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:42.075385 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:42.075385 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:42.075385 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:42.075385 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:42.075385 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:42.075385 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:42.075385 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:42.075385 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:42.075385 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:42.075385 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:42.075385 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:42.076321 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 19:06:42.076321 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 19:06:42.076321 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 19:06:42.076321 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 19:06:42.076321 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 19:06:42.076321 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 19:06:42.076321 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 19:06:42.076321 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:06:42.079945 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 498/983] 10.222.0.61 () {56 vars in 1111 bytes} [Mon Aug 22 19:06:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 19:06:42.084590 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9534d3bf-046b-4bf1-b112-03c3cae23c88 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:42.154891 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:42.161851 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def2f12f-ef92-4539-92f4-9c410a94205d None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:42.197910 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c645fd-2060-4af0-b44f-3f828087a6ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:42.198845 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c645fd-2060-4af0-b44f-3f828087a6ea None admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:42.199352 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c645fd-2060-4af0-b44f-3f828087a6ea None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:42.199635 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c645fd-2060-4af0-b44f-3f828087a6ea None admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:42.200391 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c645fd-2060-4af0-b44f-3f828087a6ea None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:42.201934 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c645fd-2060-4af0-b44f-3f828087a6ea None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:42.204550 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: WARNING keystone.server.flask.common [None req-c5c645fd-2060-4af0-b44f-3f828087a6ea None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 22 19:06:42.213207 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9534d3bf-046b-4bf1-b112-03c3cae23c88 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:42.218400 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.sql.core [None req-c5c645fd-2060-4af0-b44f-3f828087a6ea None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 22 19:06:42.218400 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 22 19:06:42.218400 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [parameters: {'user_id': 'a92fe6c0505149dc9325a9b2b287f784', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 22 19:06:42.218400 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81532) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 22 19:06:42.239910 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-c5c645fd-2060-4af0-b44f-3f828087a6ea None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 22 19:06:42.239910 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:42.239910 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 22 19:06:42.239910 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 22 19:06:42.239910 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 22 19:06:42.239910 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 22 19:06:42.239910 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 22 19:06:42.239910 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application result = self._query(query) Aug 22 19:06:42.239910 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 22 19:06:42.239910 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application conn.query(q) Aug 22 19:06:42.239910 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 22 19:06:42.239910 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 22 19:06:42.239910 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 22 19:06:42.239910 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application result.read() Aug 22 19:06:42.239910 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 22 19:06:42.240857 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 22 19:06:42.240857 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 22 19:06:42.240857 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 22 19:06:42.240857 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 22 19:06:42.240857 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 22 19:06:42.240857 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 22 19:06:42.240857 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 22 19:06:42.240857 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 22 19:06:42.240857 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:06:42.240857 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 22 19:06:42.240857 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:06:42.240857 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:42.240857 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 22 19:06:42.240857 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 22 19:06:42.240857 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 22 19:06:42.240857 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 22 19:06:42.240857 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 22 19:06:42.240857 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application next(self.gen) Aug 22 19:06:42.241846 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 22 19:06:42.241846 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application yield resource Aug 22 19:06:42.241846 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 22 19:06:42.241846 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application next(self.gen) Aug 22 19:06:42.241846 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 22 19:06:42.241846 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self.session.rollback() Aug 22 19:06:42.241846 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 22 19:06:42.241846 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self.force_reraise() Aug 22 19:06:42.241846 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 22 19:06:42.241846 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise self.value Aug 22 19:06:42.241846 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 22 19:06:42.241846 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 22 19:06:42.241846 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 22 19:06:42.241846 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application session.commit() Aug 22 19:06:42.241846 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 22 19:06:42.241846 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 22 19:06:42.241846 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 22 19:06:42.242778 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self._prepare_impl() Aug 22 19:06:42.242778 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 22 19:06:42.242778 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self.session.flush() Aug 22 19:06:42.242778 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 22 19:06:42.242778 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self._flush(objects) Aug 22 19:06:42.242778 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 22 19:06:42.242778 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 22 19:06:42.242778 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 22 19:06:42.242778 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application compat.raise_( Aug 22 19:06:42.242778 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 22 19:06:42.242778 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception Aug 22 19:06:42.242778 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 22 19:06:42.242778 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application flush_context.execute() Aug 22 19:06:42.242778 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 22 19:06:42.242778 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rec.execute(self) Aug 22 19:06:42.242778 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 22 19:06:42.242778 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 22 19:06:42.244399 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 22 19:06:42.244399 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 22 19:06:42.244399 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 22 19:06:42.244399 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 22 19:06:42.244399 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 22 19:06:42.244399 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 22 19:06:42.244399 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 22 19:06:42.244399 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 22 19:06:42.244399 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 22 19:06:42.244399 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 22 19:06:42.244399 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 22 19:06:42.244399 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 22 19:06:42.244399 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 22 19:06:42.244399 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 22 19:06:42.244399 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 22 19:06:42.244399 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception Aug 22 19:06:42.245253 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 22 19:06:42.245253 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 22 19:06:42.245253 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 22 19:06:42.245253 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 22 19:06:42.245253 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 22 19:06:42.245253 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application result = self._query(query) Aug 22 19:06:42.245253 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 22 19:06:42.245253 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application conn.query(q) Aug 22 19:06:42.245253 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 22 19:06:42.245253 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 22 19:06:42.245253 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 22 19:06:42.245253 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application result.read() Aug 22 19:06:42.245253 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 22 19:06:42.245253 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 22 19:06:42.245253 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 22 19:06:42.245253 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 22 19:06:42.246703 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 22 19:06:42.246703 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 22 19:06:42.246703 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 22 19:06:42.246703 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 22 19:06:42.246703 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 22 19:06:42.246703 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 22 19:06:42.246703 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [parameters: {'user_id': 'a92fe6c0505149dc9325a9b2b287f784', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 22 19:06:42.246703 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 22 19:06:42.246703 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:06:42.246703 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 19:06:42.246703 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:06:42.246703 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:42.246703 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:42.246703 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:42.246703 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:42.248398 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:42.248398 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:42.248398 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:42.248398 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:42.248398 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:42.248398 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:42.248398 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:42.248398 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 22 19:06:42.248398 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 22 19:06:42.248398 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:42.248398 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:42.248398 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 19:06:42.248398 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 19:06:42.248398 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 19:06:42.248398 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 19:06:42.248398 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 22 19:06:42.248398 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 22 19:06:42.249946 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 22 19:06:42.249946 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 22 19:06:42.249946 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 22 19:06:42.249946 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 22 19:06:42.249946 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 22 19:06:42.249946 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:06:42.249946 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 486/984] 10.222.0.61 () {60 vars in 1131 bytes} [Mon Aug 22 19:06:42 2022] POST /identity/v3/users => generated 160 bytes in 207 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 22 19:06:42.249946 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c645fd-2060-4af0-b44f-3f828087a6ea None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:42.265634 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12791107-6d7e-4364-a011-4c606f7cb5bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:42.266278 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12791107-6d7e-4364-a011-4c606f7cb5bf None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:42.266556 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12791107-6d7e-4364-a011-4c606f7cb5bf None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:42.266825 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12791107-6d7e-4364-a011-4c606f7cb5bf None admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:42.267522 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12791107-6d7e-4364-a011-4c606f7cb5bf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:42.267867 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12791107-6d7e-4364-a011-4c606f7cb5bf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:42.269380 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12791107-6d7e-4364-a011-4c606f7cb5bf None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:42.286884 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 499/985] 10.222.0.61 () {56 vars in 1148 bytes} [Mon Aug 22 19:06:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:42.286884 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12791107-6d7e-4364-a011-4c606f7cb5bf None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:42.352836 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:42.363402 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c645fd-2060-4af0-b44f-3f828087a6ea None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:42.396320 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12791107-6d7e-4364-a011-4c606f7cb5bf None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:42.398875 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ba9f55-9011-4786-96d2-f875302ada38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:42.399506 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ba9f55-9011-4786-96d2-f875302ada38 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:42.399813 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ba9f55-9011-4786-96d2-f875302ada38 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:42.400081 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ba9f55-9011-4786-96d2-f875302ada38 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:42.408541 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ba9f55-9011-4786-96d2-f875302ada38 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:42.410230 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ba9f55-9011-4786-96d2-f875302ada38 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:42.422199 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-51ba9f55-9011-4786-96d2-f875302ada38 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 22 19:06:42.422199 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:42.422199 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:42.422199 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:42.422199 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:42.422199 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:42.422199 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:42.422199 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:42.422199 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:42.422199 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:42.422199 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:42.422199 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:42.422199 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 19:06:42.422199 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 19:06:42.422199 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 19:06:42.422199 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 19:06:42.423203 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:42.423203 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:42.423203 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 19:06:42.423203 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 19:06:42.423203 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 19:06:42.423203 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 19:06:42.423203 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:42.423203 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:42.423203 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:42.423203 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:42.423203 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:42.423203 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:42.423203 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:42.423203 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:42.423203 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:42.423203 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:42.423203 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:42.423203 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:42.424207 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:42.424207 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:42.424207 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 19:06:42.424207 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 19:06:42.424207 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 19:06:42.424207 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 19:06:42.424207 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 19:06:42.424207 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 19:06:42.424207 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 22 19:06:42.424207 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:06:42.424207 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 487/986] 10.222.0.61 () {56 vars in 1099 bytes} [Mon Aug 22 19:06:42 2022] GET /identity/v3/users/cinder => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:42.424207 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ba9f55-9011-4786-96d2-f875302ada38 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:42.436754 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b34f75-619b-4932-8009-ba4785d03b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:42.437368 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b34f75-619b-4932-8009-ba4785d03b10 None admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:42.437653 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b34f75-619b-4932-8009-ba4785d03b10 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:42.437915 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b34f75-619b-4932-8009-ba4785d03b10 None admin] PATH_INFO: `/v3/services` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:42.438557 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b34f75-619b-4932-8009-ba4785d03b10 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:42.439927 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b34f75-619b-4932-8009-ba4785d03b10 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:42.443747 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:42.456822 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 500/987] 10.222.0.61 () {60 vars in 1141 bytes} [Mon Aug 22 19:06:42 2022] POST /identity/v3/services => generated 242 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:06:42.456822 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b34f75-619b-4932-8009-ba4785d03b10 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:42.560082 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ba9f55-9011-4786-96d2-f875302ada38 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:42.613449 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19f825b-78e4-4320-a8b2-f5db13c47c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:42.614331 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19f825b-78e4-4320-a8b2-f5db13c47c59 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:42.614610 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19f825b-78e4-4320-a8b2-f5db13c47c59 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:42.614877 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19f825b-78e4-4320-a8b2-f5db13c47c59 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:42.615608 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19f825b-78e4-4320-a8b2-f5db13c47c59 None admin] RBAC: Adding query filter params (role.id=aa9eb04acf884c51ad2adb05380bc7a1, scope.project.id=e07dc5ebdc504c6da706a2d21c0a3970, user.id=399396a8fe254da199bc48b0ca5243d0) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:42.615981 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19f825b-78e4-4320-a8b2-f5db13c47c59 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:42.617396 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19f825b-78e4-4320-a8b2-f5db13c47c59 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:42.627356 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 488/988] 10.222.0.61 () {56 vars in 1374 bytes} [Mon Aug 22 19:06:42 2022] GET /identity/v3/role_assignments?user.id=399396a8fe254da199bc48b0ca5243d0&scope.project.id=e07dc5ebdc504c6da706a2d21c0a3970&role.id=aa9eb04acf884c51ad2adb05380bc7a1 => generated 608 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:42.627580 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b34f75-619b-4932-8009-ba4785d03b10 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:42.645355 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:42.672108 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1334c0-04df-4a83-a602-c7507d948b03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:42.672640 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1334c0-04df-4a83-a602-c7507d948b03 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:42.672916 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1334c0-04df-4a83-a602-c7507d948b03 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:42.673185 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1334c0-04df-4a83-a602-c7507d948b03 None admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:42.673902 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1334c0-04df-4a83-a602-c7507d948b03 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:42.674206 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1334c0-04df-4a83-a602-c7507d948b03 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:42.675693 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1334c0-04df-4a83-a602-c7507d948b03 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:42.716482 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 501/989] 10.222.0.61 () {56 vars in 1101 bytes} [Mon Aug 22 19:06:42 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:43.426091 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2442bbed-b98a-4a89-9af0-fc766b44f2e1 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:43.426596 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2442bbed-b98a-4a89-9af0-fc766b44f2e1 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:43.426596 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2442bbed-b98a-4a89-9af0-fc766b44f2e1 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:43.427975 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 489/990] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:06:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:06:43.433073 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdecee98-489a-42c7-861f-eba97f67ad4c None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:43.433548 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdecee98-489a-42c7-861f-eba97f67ad4c None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:43.433633 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdecee98-489a-42c7-861f-eba97f67ad4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:43.452147 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-cdecee98-489a-42c7-861f-eba97f67ad4c None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:06:43.453351 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-cdecee98-489a-42c7-861f-eba97f67ad4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:06:43.525449 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 502/991] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:06:43 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 19:06:43.794398 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2442bbed-b98a-4a89-9af0-fc766b44f2e1 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:43.875188 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:43.898482 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2442bbed-b98a-4a89-9af0-fc766b44f2e1 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:43.934287 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681facdd-0850-44d1-aef6-681249c2cd41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:43.934912 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681facdd-0850-44d1-aef6-681249c2cd41 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:43.935190 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681facdd-0850-44d1-aef6-681249c2cd41 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:43.935459 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681facdd-0850-44d1-aef6-681249c2cd41 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:43.936098 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681facdd-0850-44d1-aef6-681249c2cd41 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:43.937455 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681facdd-0850-44d1-aef6-681249c2cd41 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:43.946015 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-681facdd-0850-44d1-aef6-681249c2cd41 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 22 19:06:43.946015 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:43.946015 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 19:06:43.946015 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 19:06:43.946015 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 19:06:43.946015 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 19:06:43.946015 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 19:06:43.946015 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 19:06:43.946015 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 22 19:06:43.946015 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:06:43.946015 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 19:06:43.946015 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:06:43.946015 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:43.946015 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:43.946015 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:43.946015 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:43.946015 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:43.947026 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:43.947026 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:43.947026 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:43.947026 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:43.947026 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:43.947026 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:43.947026 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 19:06:43.947026 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 19:06:43.947026 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 19:06:43.947026 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 19:06:43.947026 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:43.947026 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:43.947026 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:43.947026 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:43.947026 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:43.947026 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:43.948307 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:43.948307 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:43.948307 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:43.948307 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:43.948307 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:43.948307 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:43.948307 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:43.948307 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:43.948307 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:43.948307 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:43.948307 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 19:06:43.948307 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 19:06:43.948307 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 22 19:06:43.948307 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:06:43.952487 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 490/992] 10.222.0.61 () {56 vars in 1126 bytes} [Mon Aug 22 19:06:43 2022] GET /identity/v3/services/object-store => generated 93 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:43.953476 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdecee98-489a-42c7-861f-eba97f67ad4c None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:44.052837 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdecee98-489a-42c7-861f-eba97f67ad4c None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:44.054299 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:44.088601 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98006c02-4380-4379-aaeb-7a65eb956705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:44.089214 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98006c02-4380-4379-aaeb-7a65eb956705 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:44.089492 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98006c02-4380-4379-aaeb-7a65eb956705 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:44.089765 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98006c02-4380-4379-aaeb-7a65eb956705 None admin] PATH_INFO: `/v3/services` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:44.090399 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98006c02-4380-4379-aaeb-7a65eb956705 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:44.090706 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98006c02-4380-4379-aaeb-7a65eb956705 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:44.092100 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98006c02-4380-4379-aaeb-7a65eb956705 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:44.097602 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 503/993] 10.222.0.61 () {56 vars in 1122 bytes} [Mon Aug 22 19:06:43 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:44.103213 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681facdd-0850-44d1-aef6-681249c2cd41 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:44.212027 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681facdd-0850-44d1-aef6-681249c2cd41 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:44.245634 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c653601a-7e53-4b1e-8167-10c3586ede93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:44.246238 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c653601a-7e53-4b1e-8167-10c3586ede93 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:44.246514 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c653601a-7e53-4b1e-8167-10c3586ede93 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:44.246784 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c653601a-7e53-4b1e-8167-10c3586ede93 None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:44.247413 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c653601a-7e53-4b1e-8167-10c3586ede93 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:44.247745 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c653601a-7e53-4b1e-8167-10c3586ede93 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:44.249147 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c653601a-7e53-4b1e-8167-10c3586ede93 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:44.254341 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 491/994] 10.222.0.61 () {56 vars in 1122 bytes} [Mon Aug 22 19:06:44 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:44.259951 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98006c02-4380-4379-aaeb-7a65eb956705 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:44.360310 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98006c02-4380-4379-aaeb-7a65eb956705 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:44.394068 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05474a2-bb97-44ec-a7eb-ded7c71d0d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:44.394676 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05474a2-bb97-44ec-a7eb-ded7c71d0d63 None admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:44.394949 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05474a2-bb97-44ec-a7eb-ded7c71d0d63 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:44.395218 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05474a2-bb97-44ec-a7eb-ded7c71d0d63 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:44.395914 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05474a2-bb97-44ec-a7eb-ded7c71d0d63 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:44.397271 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05474a2-bb97-44ec-a7eb-ded7c71d0d63 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:44.414159 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 504/995] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 19:06:44 2022] POST /identity/v3/endpoints => generated 325 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:06:44.436886 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397e7062-71fb-4f5b-b3f8-a6799c5539ca None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:44.437437 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397e7062-71fb-4f5b-b3f8-a6799c5539ca None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:44.437437 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397e7062-71fb-4f5b-b3f8-a6799c5539ca None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:44.439012 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 492/996] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:06:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:06:44.444873 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c6c631-3970-4216-90f5-4d47aa9e8f98 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:44.445386 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c6c631-3970-4216-90f5-4d47aa9e8f98 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:44.445463 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c6c631-3970-4216-90f5-4d47aa9e8f98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:44.465206 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-a4c6c631-3970-4216-90f5-4d47aa9e8f98 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:06:44.466442 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-a4c6c631-3970-4216-90f5-4d47aa9e8f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:06:44.489975 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:44.550821 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 505/997] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:06:44 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 19:06:44.573879 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11c7931-77cb-4235-8642-7bfe4db979d6 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:44.574160 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11c7931-77cb-4235-8642-7bfe4db979d6 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:44.574446 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11c7931-77cb-4235-8642-7bfe4db979d6 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:44.575955 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 493/998] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:06:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:06:44.581518 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1beb6f4-e11f-4e21-8065-b7832c4c2849 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:44.581817 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1beb6f4-e11f-4e21-8065-b7832c4c2849 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:44.582095 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1beb6f4-e11f-4e21-8065-b7832c4c2849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:44.600292 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-d1beb6f4-e11f-4e21-8065-b7832c4c2849 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:06:44.601506 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-d1beb6f4-e11f-4e21-8065-b7832c4c2849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:06:44.669649 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 506/999] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:06:44 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 19:06:44.814556 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11c7931-77cb-4235-8642-7bfe4db979d6 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:44.824614 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:44.919086 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11c7931-77cb-4235-8642-7bfe4db979d6 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:44.936384 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1beb6f4-e11f-4e21-8065-b7832c4c2849 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:44.953975 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423fbab0-282d-47eb-9959-b046cf73f8d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:44.954613 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423fbab0-282d-47eb-9959-b046cf73f8d2 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:44.954894 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423fbab0-282d-47eb-9959-b046cf73f8d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:44.955169 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423fbab0-282d-47eb-9959-b046cf73f8d2 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:44.955797 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423fbab0-282d-47eb-9959-b046cf73f8d2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:44.957160 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423fbab0-282d-47eb-9959-b046cf73f8d2 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:44.966964 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-423fbab0-282d-47eb-9959-b046cf73f8d2 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 22 19:06:44.966964 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:44.966964 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 19:06:44.966964 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 19:06:44.966964 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 19:06:44.966964 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 19:06:44.966964 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 19:06:44.966964 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 19:06:44.966964 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 22 19:06:44.966964 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:06:44.966964 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 19:06:44.966964 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:06:44.966964 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:44.966964 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:44.966964 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:44.966964 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:44.966964 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:44.968752 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:44.968752 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:44.968752 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:44.968752 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:44.968752 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:44.968752 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:44.968752 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 19:06:44.968752 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 19:06:44.968752 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 19:06:44.968752 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 19:06:44.968752 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:44.968752 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:44.968752 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:44.968752 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:44.968752 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:44.968752 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:44.969628 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:44.969628 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:44.969628 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:44.969628 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:44.969628 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:44.969628 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:44.969628 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:44.969628 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:44.969628 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:44.969628 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:44.969628 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 19:06:44.969628 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 19:06:44.969628 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 22 19:06:44.969628 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:06:44.969628 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 494/1000] 10.222.0.61 () {56 vars in 1111 bytes} [Mon Aug 22 19:06:44 2022] GET /identity/v3/services/compute => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:44.975076 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423fbab0-282d-47eb-9959-b046cf73f8d2 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:45.026133 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:45.047134 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1beb6f4-e11f-4e21-8065-b7832c4c2849 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:45.048326 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:45.081414 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423fbab0-282d-47eb-9959-b046cf73f8d2 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:45.083781 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c8e973-f8d9-4b36-867e-3531916e7118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:45.084424 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c8e973-f8d9-4b36-867e-3531916e7118 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:45.084701 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c8e973-f8d9-4b36-867e-3531916e7118 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:45.084971 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c8e973-f8d9-4b36-867e-3531916e7118 None admin] PATH_INFO: `/v3/services/image` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:45.085566 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c8e973-f8d9-4b36-867e-3531916e7118 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:45.086928 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c8e973-f8d9-4b36-867e-3531916e7118 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:45.097704 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-18c8e973-f8d9-4b36-867e-3531916e7118 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 22 19:06:45.097704 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:45.097704 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 19:06:45.097704 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 19:06:45.097704 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 19:06:45.097704 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 19:06:45.097704 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 19:06:45.097704 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 19:06:45.097704 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 22 19:06:45.097704 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:06:45.097704 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 19:06:45.097704 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:06:45.097704 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:45.097704 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:45.097704 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:45.097704 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:45.097704 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:45.098724 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:45.098724 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:45.098724 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:45.098724 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:45.098724 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:45.098724 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:45.098724 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 19:06:45.098724 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 19:06:45.098724 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 19:06:45.098724 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 19:06:45.098724 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:45.098724 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:45.098724 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:45.098724 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:45.098724 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:45.098724 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:45.099591 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:45.099591 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:45.099591 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:45.099591 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:45.099591 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:45.099591 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:45.099591 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:45.099591 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:45.099591 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:45.099591 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:45.099591 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 19:06:45.099591 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 19:06:45.099591 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 22 19:06:45.099591 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:06:45.099591 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 507/1001] 10.222.0.61 () {56 vars in 1105 bytes} [Mon Aug 22 19:06:44 2022] GET /identity/v3/services/image => generated 86 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:45.104512 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c8e973-f8d9-4b36-867e-3531916e7118 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:45.124502 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa33e2cf-3759-4f4d-9780-778d8f8ef723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:45.125107 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa33e2cf-3759-4f4d-9780-778d8f8ef723 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:45.125381 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa33e2cf-3759-4f4d-9780-778d8f8ef723 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:45.125646 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa33e2cf-3759-4f4d-9780-778d8f8ef723 None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:45.126274 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa33e2cf-3759-4f4d-9780-778d8f8ef723 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:45.126569 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa33e2cf-3759-4f4d-9780-778d8f8ef723 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:45.127938 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa33e2cf-3759-4f4d-9780-778d8f8ef723 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:45.133004 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 495/1002] 10.222.0.61 () {56 vars in 1112 bytes} [Mon Aug 22 19:06:44 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 19:06:45.138467 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa33e2cf-3759-4f4d-9780-778d8f8ef723 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:45.212662 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c8e973-f8d9-4b36-867e-3531916e7118 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:45.246683 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa33e2cf-3759-4f4d-9780-778d8f8ef723 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:45.248590 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dda78e2-2785-4d71-86b9-98e6eff754d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:45.249373 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dda78e2-2785-4d71-86b9-98e6eff754d2 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:45.249643 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dda78e2-2785-4d71-86b9-98e6eff754d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:45.249915 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dda78e2-2785-4d71-86b9-98e6eff754d2 None admin] PATH_INFO: `/v3/services` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:45.250533 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dda78e2-2785-4d71-86b9-98e6eff754d2 None admin] RBAC: Adding query filter params (name=image) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:45.250828 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dda78e2-2785-4d71-86b9-98e6eff754d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:45.252217 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dda78e2-2785-4d71-86b9-98e6eff754d2 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:45.257115 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 508/1003] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:06:45 2022] GET /identity/v3/services?name=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 19:06:45.262547 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dda78e2-2785-4d71-86b9-98e6eff754d2 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:45.274979 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:45.285302 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f109580c-fb93-4325-9442-2227c1756515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:45.286133 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f109580c-fb93-4325-9442-2227c1756515 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:45.286303 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f109580c-fb93-4325-9442-2227c1756515 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:45.286445 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f109580c-fb93-4325-9442-2227c1756515 None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:45.287067 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f109580c-fb93-4325-9442-2227c1756515 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:45.287369 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f109580c-fb93-4325-9442-2227c1756515 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:45.288797 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f109580c-fb93-4325-9442-2227c1756515 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:45.290875 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:45.299884 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 496/1004] 10.222.0.61 () {56 vars in 1112 bytes} [Mon Aug 22 19:06:45 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:45.300910 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f109580c-fb93-4325-9442-2227c1756515 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:45.374073 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dda78e2-2785-4d71-86b9-98e6eff754d2 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:45.406589 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f109580c-fb93-4325-9442-2227c1756515 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:45.409425 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18d72c8-c9f7-4cbc-9050-5e37b7eb53ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:45.410037 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18d72c8-c9f7-4cbc-9050-5e37b7eb53ad None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:45.410316 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18d72c8-c9f7-4cbc-9050-5e37b7eb53ad None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:45.410587 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18d72c8-c9f7-4cbc-9050-5e37b7eb53ad None admin] PATH_INFO: `/v3/services` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:45.411219 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18d72c8-c9f7-4cbc-9050-5e37b7eb53ad None admin] RBAC: Adding query filter params (type=image) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:45.411522 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18d72c8-c9f7-4cbc-9050-5e37b7eb53ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:45.412951 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18d72c8-c9f7-4cbc-9050-5e37b7eb53ad None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:45.427689 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 509/1005] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:06:45 2022] GET /identity/v3/services?type=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 19:06:45.454452 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20cf4ee-a682-489a-9b87-97a7f13d8634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:45.455109 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20cf4ee-a682-489a-9b87-97a7f13d8634 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:45.455348 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20cf4ee-a682-489a-9b87-97a7f13d8634 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:45.455623 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20cf4ee-a682-489a-9b87-97a7f13d8634 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:45.456377 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20cf4ee-a682-489a-9b87-97a7f13d8634 None admin] RBAC: Adding query filter params (interface=public, service_id=9ac4b9c9d58e4d609d6f552f50028c09, region_id=RegionOne) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:45.456685 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20cf4ee-a682-489a-9b87-97a7f13d8634 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:45.458064 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20cf4ee-a682-489a-9b87-97a7f13d8634 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:45.464111 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 497/1006] 10.222.0.61 () {56 vars in 1251 bytes} [Mon Aug 22 19:06:45 2022] GET /identity/v3/endpoints?service_id=9ac4b9c9d58e4d609d6f552f50028c09&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:45.470395 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18d72c8-c9f7-4cbc-9050-5e37b7eb53ad None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:45.556655 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:45.584188 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18d72c8-c9f7-4cbc-9050-5e37b7eb53ad None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:45.619393 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bce741d-a95f-4d65-a903-6596bec34813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:45.620043 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bce741d-a95f-4d65-a903-6596bec34813 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:45.620321 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bce741d-a95f-4d65-a903-6596bec34813 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:45.620592 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bce741d-a95f-4d65-a903-6596bec34813 None admin] PATH_INFO: `/v3/services` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:45.621201 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bce741d-a95f-4d65-a903-6596bec34813 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:45.622555 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bce741d-a95f-4d65-a903-6596bec34813 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:45.629197 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 510/1007] 10.222.0.61 () {56 vars in 1087 bytes} [Mon Aug 22 19:06:45 2022] GET /identity/v3/services => generated 1727 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:06:46.357815 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb76104-db59-4e9c-b35c-019cd5a77876 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:46.358177 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb76104-db59-4e9c-b35c-019cd5a77876 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:46.358415 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb76104-db59-4e9c-b35c-019cd5a77876 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:46.359907 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 498/1008] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:06:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:06:46.365275 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b0a0c4-2c65-461b-b0d1-1b13cf02f1e1 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:46.365566 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b0a0c4-2c65-461b-b0d1-1b13cf02f1e1 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:46.365843 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b0a0c4-2c65-461b-b0d1-1b13cf02f1e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:46.384176 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-c7b0a0c4-2c65-461b-b0d1-1b13cf02f1e1 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:06:46.385374 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-c7b0a0c4-2c65-461b-b0d1-1b13cf02f1e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:06:46.454639 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 511/1009] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:06:46 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 19:06:46.721538 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb76104-db59-4e9c-b35c-019cd5a77876 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:46.787274 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:46.828583 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb76104-db59-4e9c-b35c-019cd5a77876 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:46.862387 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e71bc97-5d46-4e05-9438-2ad871f58d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:46.863026 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e71bc97-5d46-4e05-9438-2ad871f58d6f None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:46.863304 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e71bc97-5d46-4e05-9438-2ad871f58d6f None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:46.863574 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e71bc97-5d46-4e05-9438-2ad871f58d6f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:46.864984 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e71bc97-5d46-4e05-9438-2ad871f58d6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:46.866431 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e71bc97-5d46-4e05-9438-2ad871f58d6f None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:46.871689 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 499/1010] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:06:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:46.874175 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b0a0c4-2c65-461b-b0d1-1b13cf02f1e1 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:46.942568 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:46.978191 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b0a0c4-2c65-461b-b0d1-1b13cf02f1e1 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:47.013040 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b4d9b6-9bd0-4c90-b81d-d2fee5582560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:47.013661 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b4d9b6-9bd0-4c90-b81d-d2fee5582560 None admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:47.013937 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b4d9b6-9bd0-4c90-b81d-d2fee5582560 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:47.014768 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b4d9b6-9bd0-4c90-b81d-d2fee5582560 None admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:47.015424 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b4d9b6-9bd0-4c90-b81d-d2fee5582560 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:47.016927 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b4d9b6-9bd0-4c90-b81d-d2fee5582560 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:47.045522 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 512/1011] 10.222.0.61 () {60 vars in 1141 bytes} [Mon Aug 22 19:06:46 2022] POST /identity/v3/projects => generated 311 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:06:47.318625 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bcfb67-00ad-4006-99da-aa0bc9e93d9d None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:47.318802 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bcfb67-00ad-4006-99da-aa0bc9e93d9d None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:47.319075 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bcfb67-00ad-4006-99da-aa0bc9e93d9d None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:47.320594 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 500/1012] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:06:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:06:47.325805 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be76da0-3352-4a5b-8714-1a8942723580 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:47.326096 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be76da0-3352-4a5b-8714-1a8942723580 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:47.326367 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be76da0-3352-4a5b-8714-1a8942723580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:47.344401 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-8be76da0-3352-4a5b-8714-1a8942723580 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:06:47.345643 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-8be76da0-3352-4a5b-8714-1a8942723580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:06:47.412865 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 513/1013] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:06:47 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 19:06:47.526054 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1940834-8e92-4afa-a78b-b38af8cb75cc None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:47.526418 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1940834-8e92-4afa-a78b-b38af8cb75cc None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:47.526642 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1940834-8e92-4afa-a78b-b38af8cb75cc None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:47.528152 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 501/1014] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:06:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:06:47.533733 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4755f42-89dc-4a3c-ba6a-9c82456d464a None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:47.534019 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4755f42-89dc-4a3c-ba6a-9c82456d464a None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:47.534295 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4755f42-89dc-4a3c-ba6a-9c82456d464a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:47.552926 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-c4755f42-89dc-4a3c-ba6a-9c82456d464a None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:06:47.554120 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-c4755f42-89dc-4a3c-ba6a-9c82456d464a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:06:47.601848 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:47.625165 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 514/1015] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:06:47 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 19:06:47.680592 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1940834-8e92-4afa-a78b-b38af8cb75cc None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:47.778639 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1940834-8e92-4afa-a78b-b38af8cb75cc None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:47.807233 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:47.813966 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9b94e5-8e2e-48bf-9b9d-ef7effb888cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:47.814567 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9b94e5-8e2e-48bf-9b9d-ef7effb888cf None admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:47.814840 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9b94e5-8e2e-48bf-9b9d-ef7effb888cf None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:47.815107 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9b94e5-8e2e-48bf-9b9d-ef7effb888cf None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:47.815778 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9b94e5-8e2e-48bf-9b9d-ef7effb888cf None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:47.817116 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9b94e5-8e2e-48bf-9b9d-ef7effb888cf None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:47.833886 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 502/1016] 10.222.0.61 () {60 vars in 1141 bytes} [Mon Aug 22 19:06:47 2022] POST /identity/v3/services => generated 242 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:06:47.894078 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4755f42-89dc-4a3c-ba6a-9c82456d464a None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:48.001218 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4755f42-89dc-4a3c-ba6a-9c82456d464a None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:48.043162 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f661d62-dc34-4d4e-84e0-5441aa65c010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:48.044241 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f661d62-dc34-4d4e-84e0-5441aa65c010 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:48.044724 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f661d62-dc34-4d4e-84e0-5441aa65c010 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:48.045209 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f661d62-dc34-4d4e-84e0-5441aa65c010 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:48.046019 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f661d62-dc34-4d4e-84e0-5441aa65c010 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:48.047576 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f661d62-dc34-4d4e-84e0-5441aa65c010 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:48.056778 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-5f661d62-dc34-4d4e-84e0-5441aa65c010 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 22 19:06:48.056778 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:48.056778 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 19:06:48.056778 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 19:06:48.056778 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 19:06:48.056778 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 19:06:48.056778 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 19:06:48.056778 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 19:06:48.056778 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 22 19:06:48.056778 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:06:48.056778 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 19:06:48.056778 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:06:48.056778 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:48.056778 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:48.056778 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:48.056778 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:48.056778 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:48.057881 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:48.057881 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:48.057881 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:48.057881 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:48.057881 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:48.057881 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:48.057881 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 19:06:48.057881 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 19:06:48.057881 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 19:06:48.057881 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 19:06:48.057881 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:48.057881 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:48.057881 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:48.057881 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:48.057881 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:48.057881 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:48.058739 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:48.058739 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:48.058739 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:48.058739 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:48.058739 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:48.058739 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:48.058739 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:48.058739 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:48.058739 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:48.058739 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:48.058739 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 19:06:48.058739 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 19:06:48.058739 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 22 19:06:48.058739 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:06:48.059491 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 515/1017] 10.222.0.61 () {56 vars in 1111 bytes} [Mon Aug 22 19:06:47 2022] GET /identity/v3/services/compute => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:48.064599 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9b94e5-8e2e-48bf-9b9d-ef7effb888cf None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:48.175911 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9b94e5-8e2e-48bf-9b9d-ef7effb888cf None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:48.212072 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6e4469-0610-4f55-948e-22b215c11082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:48.212686 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6e4469-0610-4f55-948e-22b215c11082 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:48.212966 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6e4469-0610-4f55-948e-22b215c11082 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:48.213239 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6e4469-0610-4f55-948e-22b215c11082 None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:48.213870 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6e4469-0610-4f55-948e-22b215c11082 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:48.214174 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6e4469-0610-4f55-948e-22b215c11082 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:48.215536 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6e4469-0610-4f55-948e-22b215c11082 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:48.227193 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 503/1018] 10.222.0.61 () {56 vars in 1112 bytes} [Mon Aug 22 19:06:48 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:48.228161 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f661d62-dc34-4d4e-84e0-5441aa65c010 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:48.272974 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:48.334782 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f661d62-dc34-4d4e-84e0-5441aa65c010 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:48.368121 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19074953-bbb7-4d4f-bbad-bfea689a9b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:48.368734 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19074953-bbb7-4d4f-bbad-bfea689a9b64 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:48.369015 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19074953-bbb7-4d4f-bbad-bfea689a9b64 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:48.369290 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19074953-bbb7-4d4f-bbad-bfea689a9b64 None admin] PATH_INFO: `/v3/services` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:48.369919 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19074953-bbb7-4d4f-bbad-bfea689a9b64 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:48.370221 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19074953-bbb7-4d4f-bbad-bfea689a9b64 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:48.371584 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19074953-bbb7-4d4f-bbad-bfea689a9b64 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:48.381535 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 516/1019] 10.222.0.61 () {56 vars in 1112 bytes} [Mon Aug 22 19:06:48 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 19:06:48.382529 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6e4469-0610-4f55-948e-22b215c11082 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:48.438237 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:48.486168 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6e4469-0610-4f55-948e-22b215c11082 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:48.521120 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c0b079-8c50-42df-9dfe-40932c883fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:48.521731 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c0b079-8c50-42df-9dfe-40932c883fad None admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:48.522011 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c0b079-8c50-42df-9dfe-40932c883fad None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:48.522283 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c0b079-8c50-42df-9dfe-40932c883fad None admin] PATH_INFO: `/v3/endpoints` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:48.522945 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c0b079-8c50-42df-9dfe-40932c883fad None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:48.524335 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c0b079-8c50-42df-9dfe-40932c883fad None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:48.544807 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 504/1020] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 19:06:48 2022] POST /identity/v3/endpoints => generated 334 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 19:06:49.016786 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313b3b17-a2b7-407d-8e3f-85637070f315 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:49.017322 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313b3b17-a2b7-407d-8e3f-85637070f315 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:49.017322 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313b3b17-a2b7-407d-8e3f-85637070f315 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:49.018822 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 517/1021] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:06:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:06:49.023886 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d160898c-68d1-4306-9f40-50dc31ce44a1 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:49.024311 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d160898c-68d1-4306-9f40-50dc31ce44a1 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:49.024479 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d160898c-68d1-4306-9f40-50dc31ce44a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:49.042565 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-d160898c-68d1-4306-9f40-50dc31ce44a1 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:06:49.044124 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-d160898c-68d1-4306-9f40-50dc31ce44a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:06:49.120565 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 505/1022] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:06:49 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 19:06:49.385189 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313b3b17-a2b7-407d-8e3f-85637070f315 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:49.490088 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313b3b17-a2b7-407d-8e3f-85637070f315 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:49.492515 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:49.526643 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bbb153-2ab1-4222-9f13-5539895fbb9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:49.527277 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bbb153-2ab1-4222-9f13-5539895fbb9c None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:49.527554 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bbb153-2ab1-4222-9f13-5539895fbb9c None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:49.527854 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bbb153-2ab1-4222-9f13-5539895fbb9c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:49.528969 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bbb153-2ab1-4222-9f13-5539895fbb9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:49.530420 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bbb153-2ab1-4222-9f13-5539895fbb9c None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:49.532175 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 518/1023] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:06:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:49.537634 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d160898c-68d1-4306-9f40-50dc31ce44a1 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:49.572214 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:49.641078 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d160898c-68d1-4306-9f40-50dc31ce44a1 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:49.675689 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b311be23-c123-46ad-893d-eadf60bb8d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:49.676300 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b311be23-c123-46ad-893d-eadf60bb8d7b None admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:49.676572 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b311be23-c123-46ad-893d-eadf60bb8d7b None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:49.676840 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b311be23-c123-46ad-893d-eadf60bb8d7b None admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:49.677489 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b311be23-c123-46ad-893d-eadf60bb8d7b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:49.678904 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b311be23-c123-46ad-893d-eadf60bb8d7b None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:49.707332 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 506/1024] 10.222.0.61 () {60 vars in 1132 bytes} [Mon Aug 22 19:06:49 2022] POST /identity/v3/users => generated 285 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:06:49.743255 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f3158c-5cbe-44cf-ad61-547cdd300b01 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:49.743580 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f3158c-5cbe-44cf-ad61-547cdd300b01 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:49.743839 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f3158c-5cbe-44cf-ad61-547cdd300b01 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:49.745200 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 519/1025] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:06:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:06:49.751077 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9021a1ae-10a1-4615-b2c2-b4d0d9d66b0f None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:49.751410 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9021a1ae-10a1-4615-b2c2-b4d0d9d66b0f None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:49.751621 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9021a1ae-10a1-4615-b2c2-b4d0d9d66b0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:49.771558 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-9021a1ae-10a1-4615-b2c2-b4d0d9d66b0f None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:06:49.772796 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-9021a1ae-10a1-4615-b2c2-b4d0d9d66b0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:06:49.851754 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 507/1026] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:06:49 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 19:06:50.111565 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f3158c-5cbe-44cf-ad61-547cdd300b01 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:50.222853 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f3158c-5cbe-44cf-ad61-547cdd300b01 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:50.258242 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef07d382-6078-49ee-9467-c51db987f47d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:50.258872 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef07d382-6078-49ee-9467-c51db987f47d None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:50.259146 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef07d382-6078-49ee-9467-c51db987f47d None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:50.259416 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef07d382-6078-49ee-9467-c51db987f47d None admin] PATH_INFO: `/v3/services/image` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:50.260022 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef07d382-6078-49ee-9467-c51db987f47d None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:50.261377 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef07d382-6078-49ee-9467-c51db987f47d None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:50.270608 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-ef07d382-6078-49ee-9467-c51db987f47d None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 22 19:06:50.270608 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:50.270608 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 19:06:50.270608 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 19:06:50.270608 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 19:06:50.270608 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 19:06:50.270608 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 19:06:50.270608 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 19:06:50.270608 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 22 19:06:50.270608 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:06:50.270608 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 19:06:50.270608 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:06:50.270608 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:50.270608 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:50.270608 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:50.270608 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:50.270608 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:50.271595 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:50.271595 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:50.271595 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:50.271595 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:50.271595 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:50.271595 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:50.271595 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 19:06:50.271595 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 19:06:50.271595 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 19:06:50.271595 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 19:06:50.271595 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:50.271595 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:50.271595 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:50.271595 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:50.271595 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:50.271595 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:50.272739 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:50.272739 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:50.272739 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:50.272739 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:50.272739 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:50.272739 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:50.272739 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:50.272739 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:50.272739 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:50.272739 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:50.272739 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 19:06:50.272739 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 19:06:50.272739 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 22 19:06:50.272739 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:06:50.276625 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 520/1027] 10.222.0.61 () {56 vars in 1105 bytes} [Mon Aug 22 19:06:50 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:50.277963 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9021a1ae-10a1-4615-b2c2-b4d0d9d66b0f None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:50.301813 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:50.379511 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9021a1ae-10a1-4615-b2c2-b4d0d9d66b0f None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:50.415202 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b27a67-019f-4828-a1fe-04f234453332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:50.415895 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b27a67-019f-4828-a1fe-04f234453332 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:50.416169 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b27a67-019f-4828-a1fe-04f234453332 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:50.416433 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b27a67-019f-4828-a1fe-04f234453332 None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:50.417057 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b27a67-019f-4828-a1fe-04f234453332 None admin] RBAC: Adding query filter params (name=image) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:50.417352 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b27a67-019f-4828-a1fe-04f234453332 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:50.418700 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b27a67-019f-4828-a1fe-04f234453332 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:50.428296 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 508/1028] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:06:50 2022] GET /identity/v3/services?name=image => generated 122 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:50.429493 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef07d382-6078-49ee-9467-c51db987f47d None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:50.466951 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:50.537428 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef07d382-6078-49ee-9467-c51db987f47d None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:50.543400 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aba1ec-a945-4481-af6b-be793c569918 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:50.543713 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aba1ec-a945-4481-af6b-be793c569918 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:50.543990 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aba1ec-a945-4481-af6b-be793c569918 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:50.545517 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 509/1029] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:06:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:06:50.550109 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326b1b28-143d-4ca1-9b86-ddfe946024a5 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:50.550389 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326b1b28-143d-4ca1-9b86-ddfe946024a5 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:50.550664 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326b1b28-143d-4ca1-9b86-ddfe946024a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:50.568951 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-326b1b28-143d-4ca1-9b86-ddfe946024a5 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:06:50.570139 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-326b1b28-143d-4ca1-9b86-ddfe946024a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:06:50.573315 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7ce5af-2c1f-493d-9da9-a0cb9feb3a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:50.573929 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7ce5af-2c1f-493d-9da9-a0cb9feb3a40 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:50.574204 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7ce5af-2c1f-493d-9da9-a0cb9feb3a40 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:50.574472 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7ce5af-2c1f-493d-9da9-a0cb9feb3a40 None admin] PATH_INFO: `/v3/services` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:50.575104 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7ce5af-2c1f-493d-9da9-a0cb9feb3a40 None admin] RBAC: Adding query filter params (type=image) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:50.575404 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7ce5af-2c1f-493d-9da9-a0cb9feb3a40 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:50.577356 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7ce5af-2c1f-493d-9da9-a0cb9feb3a40 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:50.582214 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 521/1030] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:06:50 2022] GET /identity/v3/services?type=image => generated 350 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:50.588196 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7ce5af-2c1f-493d-9da9-a0cb9feb3a40 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:50.641945 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 510/1031] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:06:50 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 22 19:06:50.693344 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:50.696797 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7ce5af-2c1f-493d-9da9-a0cb9feb3a40 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:50.732849 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f713be8-faf5-4307-a9d1-1b2ed46044f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:50.733460 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f713be8-faf5-4307-a9d1-1b2ed46044f5 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:50.733740 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f713be8-faf5-4307-a9d1-1b2ed46044f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:50.734013 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f713be8-faf5-4307-a9d1-1b2ed46044f5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:50.734714 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f713be8-faf5-4307-a9d1-1b2ed46044f5 None admin] RBAC: Adding query filter params (interface=public, service_id=58d3be3f56fa4345974fcadcbfbf01ec, region_id=RegionOne) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:50.735021 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f713be8-faf5-4307-a9d1-1b2ed46044f5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:50.736420 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f713be8-faf5-4307-a9d1-1b2ed46044f5 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:50.746432 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 522/1032] 10.222.0.61 () {56 vars in 1251 bytes} [Mon Aug 22 19:06:50 2022] GET /identity/v3/endpoints?service_id=58d3be3f56fa4345974fcadcbfbf01ec&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 19:06:50.747721 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326b1b28-143d-4ca1-9b86-ddfe946024a5 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:50.766712 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:50.851454 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326b1b28-143d-4ca1-9b86-ddfe946024a5 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:50.886835 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e50f88-f1ef-4a59-9d1a-1f565ec75da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:50.887444 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e50f88-f1ef-4a59-9d1a-1f565ec75da5 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:50.887747 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e50f88-f1ef-4a59-9d1a-1f565ec75da5 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:50.888028 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e50f88-f1ef-4a59-9d1a-1f565ec75da5 None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:50.888633 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e50f88-f1ef-4a59-9d1a-1f565ec75da5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:50.890057 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e50f88-f1ef-4a59-9d1a-1f565ec75da5 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:50.903847 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 511/1033] 10.222.0.61 () {56 vars in 1087 bytes} [Mon Aug 22 19:06:50 2022] GET /identity/v3/services => generated 1957 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:06:50.914041 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f713be8-faf5-4307-a9d1-1b2ed46044f5 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:51.025321 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f713be8-faf5-4307-a9d1-1b2ed46044f5 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:51.061118 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111e94df-bde8-47a7-9243-140627233b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:51.061757 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111e94df-bde8-47a7-9243-140627233b34 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:51.062040 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111e94df-bde8-47a7-9243-140627233b34 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:51.062311 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111e94df-bde8-47a7-9243-140627233b34 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:51.072631 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111e94df-bde8-47a7-9243-140627233b34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 19:06:51.078258 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-111e94df-bde8-47a7-9243-140627233b34 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 22 19:06:51.078258 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:51.078258 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:51.078258 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:51.078258 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:51.078258 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:51.078258 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:51.078258 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:51.078258 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:51.078258 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:51.078258 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:51.078258 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:51.078258 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 19:06:51.078258 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 19:06:51.078258 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 19:06:51.078258 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 19:06:51.079486 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 19:06:51.079486 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 19:06:51.079486 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 19:06:51.079486 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 19:06:51.079486 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:51.079486 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:51.079486 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:51.079486 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:51.079486 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:51.079486 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:51.079486 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:51.079486 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:51.079486 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:51.079486 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:51.079486 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:51.079486 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:51.080718 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:51.080718 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:51.080718 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:51.080718 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:51.080718 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 19:06:51.080718 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 19:06:51.080718 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 19:06:51.080718 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 19:06:51.080718 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 19:06:51.080718 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 19:06:51.080718 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 22 19:06:51.080718 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:06:51.080718 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 523/1034] 10.222.0.61 () {56 vars in 1120 bytes} [Mon Aug 22 19:06:50 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:51.085264 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e50f88-f1ef-4a59-9d1a-1f565ec75da5 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:51.179073 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:51.199198 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e50f88-f1ef-4a59-9d1a-1f565ec75da5 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:51.234388 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccee9d8f-f547-425d-a67c-a8c2986819ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:51.235001 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccee9d8f-f547-425d-a67c-a8c2986819ba None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:51.235279 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccee9d8f-f547-425d-a67c-a8c2986819ba None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:51.235552 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccee9d8f-f547-425d-a67c-a8c2986819ba None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:51.236248 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccee9d8f-f547-425d-a67c-a8c2986819ba None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:51.236550 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccee9d8f-f547-425d-a67c-a8c2986819ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:51.237905 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccee9d8f-f547-425d-a67c-a8c2986819ba None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:51.251405 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 512/1035] 10.222.0.61 () {56 vars in 1115 bytes} [Mon Aug 22 19:06:51 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 19:06:51.252720 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111e94df-bde8-47a7-9243-140627233b34 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:51.280272 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:51.361100 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111e94df-bde8-47a7-9243-140627233b34 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:51.396435 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64af4424-af70-4daf-8f4a-5b57122280cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:51.397067 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64af4424-af70-4daf-8f4a-5b57122280cc None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:51.397344 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64af4424-af70-4daf-8f4a-5b57122280cc None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:51.397614 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64af4424-af70-4daf-8f4a-5b57122280cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:51.407587 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64af4424-af70-4daf-8f4a-5b57122280cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:51.409108 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64af4424-af70-4daf-8f4a-5b57122280cc None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:51.423515 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-64af4424-af70-4daf-8f4a-5b57122280cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:06:51.423515 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:51.423515 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:51.423515 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:51.423515 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:51.423515 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:51.423515 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:51.423515 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:51.423515 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:51.423515 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:51.423515 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:51.423515 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:51.423515 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 19:06:51.423515 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 19:06:51.423515 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 19:06:51.423515 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 19:06:51.424572 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:51.424572 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:51.424572 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:51.424572 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:51.424572 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:51.424572 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:51.424572 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:51.424572 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:51.424572 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:51.424572 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:51.424572 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:51.424572 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:51.424572 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:51.424572 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:51.424572 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:51.424572 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:51.424572 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 19:06:51.425498 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 19:06:51.425498 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:06:51.425498 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:06:51.427967 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 524/1036] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:06:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:51.430482 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccee9d8f-f547-425d-a67c-a8c2986819ba None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:51.534730 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccee9d8f-f547-425d-a67c-a8c2986819ba None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:51.568396 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458636c0-0330-40ad-8142-d4d88afda764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:51.569003 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458636c0-0330-40ad-8142-d4d88afda764 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:51.569281 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458636c0-0330-40ad-8142-d4d88afda764 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:51.569552 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458636c0-0330-40ad-8142-d4d88afda764 None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:51.570194 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458636c0-0330-40ad-8142-d4d88afda764 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:51.570494 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458636c0-0330-40ad-8142-d4d88afda764 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:51.571892 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458636c0-0330-40ad-8142-d4d88afda764 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:51.589318 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 513/1037] 10.222.0.61 () {56 vars in 1109 bytes} [Mon Aug 22 19:06:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:51.590647 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64af4424-af70-4daf-8f4a-5b57122280cc None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:51.668329 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:51.677320 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab127f3e-fc65-41bb-ae2e-164218667393 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:51.677607 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab127f3e-fc65-41bb-ae2e-164218667393 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:51.677885 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab127f3e-fc65-41bb-ae2e-164218667393 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:51.679364 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 514/1038] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:06:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 19:06:51.684157 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baee7621-2308-4aae-abeb-135fa443f2e0 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:51.684373 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baee7621-2308-4aae-abeb-135fa443f2e0 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:51.684650 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baee7621-2308-4aae-abeb-135fa443f2e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:51.692279 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:51.694500 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64af4424-af70-4daf-8f4a-5b57122280cc None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:51.705023 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-baee7621-2308-4aae-abeb-135fa443f2e0 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:06:51.706221 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-baee7621-2308-4aae-abeb-135fa443f2e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:06:51.728431 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ae1ee7-7991-4dc3-903b-3627225fee8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:51.729060 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ae1ee7-7991-4dc3-903b-3627225fee8b None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:51.729338 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ae1ee7-7991-4dc3-903b-3627225fee8b None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:51.729604 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ae1ee7-7991-4dc3-903b-3627225fee8b None admin] PATH_INFO: `/v3/users/nova` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:51.738336 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ae1ee7-7991-4dc3-903b-3627225fee8b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:51.739995 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ae1ee7-7991-4dc3-903b-3627225fee8b None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:51.753306 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-35ae1ee7-7991-4dc3-903b-3627225fee8b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 22 19:06:51.753306 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:51.753306 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:51.753306 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:51.753306 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:51.753306 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:51.753306 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:51.753306 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:51.753306 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:51.753306 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:51.753306 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:51.753306 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:51.753306 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 19:06:51.753306 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 19:06:51.753306 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 19:06:51.753306 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 19:06:51.754333 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:51.754333 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:51.754333 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 19:06:51.754333 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 19:06:51.754333 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 19:06:51.754333 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 19:06:51.754333 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:51.754333 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:51.754333 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:51.754333 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:51.754333 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:51.754333 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:51.754333 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:51.754333 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:51.754333 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:51.754333 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:51.754333 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:51.754333 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:51.755306 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:51.755306 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:51.755306 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 19:06:51.755306 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 19:06:51.755306 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 19:06:51.755306 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 19:06:51.755306 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 19:06:51.755306 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 19:06:51.755306 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 22 19:06:51.755306 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:06:51.755306 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 525/1039] 10.222.0.61 () {56 vars in 1093 bytes} [Mon Aug 22 19:06:51 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:51.760045 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ae1ee7-7991-4dc3-903b-3627225fee8b None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:51.777838 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 515/1040] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:06:51 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 22 19:06:51.857373 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ae1ee7-7991-4dc3-903b-3627225fee8b None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:51.892298 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaf82f5-f292-4b1a-9db7-92a7dff8b419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:51.892912 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaf82f5-f292-4b1a-9db7-92a7dff8b419 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:51.893195 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaf82f5-f292-4b1a-9db7-92a7dff8b419 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:51.893469 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaf82f5-f292-4b1a-9db7-92a7dff8b419 None admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:51.894200 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaf82f5-f292-4b1a-9db7-92a7dff8b419 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:51.894508 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaf82f5-f292-4b1a-9db7-92a7dff8b419 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:51.896038 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaf82f5-f292-4b1a-9db7-92a7dff8b419 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:51.898688 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:51.915069 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 526/1041] 10.222.0.61 () {56 vars in 1133 bytes} [Mon Aug 22 19:06:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 19:06:51.921198 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baee7621-2308-4aae-abeb-135fa443f2e0 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:52.023889 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baee7621-2308-4aae-abeb-135fa443f2e0 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:52.043770 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaf82f5-f292-4b1a-9db7-92a7dff8b419 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:52.054377 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:52.060655 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6c75a3-e206-48b7-8a4a-3c6061ad4cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:52.061292 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6c75a3-e206-48b7-8a4a-3c6061ad4cb7 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:52.061565 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6c75a3-e206-48b7-8a4a-3c6061ad4cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:52.061833 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6c75a3-e206-48b7-8a4a-3c6061ad4cb7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:52.129723 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6c75a3-e206-48b7-8a4a-3c6061ad4cb7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:52.131151 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6c75a3-e206-48b7-8a4a-3c6061ad4cb7 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:52.157459 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaf82f5-f292-4b1a-9db7-92a7dff8b419 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:52.193405 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4225d4-f08b-4bb6-94fd-c0e750f739a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:52.194149 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4225d4-f08b-4bb6-94fd-c0e750f739a2 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:52.194426 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4225d4-f08b-4bb6-94fd-c0e750f739a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:52.194818 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4225d4-f08b-4bb6-94fd-c0e750f739a2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:52.205556 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4225d4-f08b-4bb6-94fd-c0e750f739a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 19:06:52.210310 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-4a4225d4-f08b-4bb6-94fd-c0e750f739a2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 19:06:52.210310 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:52.210310 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:52.210310 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:52.210310 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:52.210310 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:52.210310 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:52.210310 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:52.210310 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:52.210310 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:52.210310 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:52.210310 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:52.210310 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 19:06:52.210310 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 19:06:52.210310 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 19:06:52.210310 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 19:06:52.211348 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 19:06:52.211348 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 19:06:52.211348 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 19:06:52.211348 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 19:06:52.211348 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:52.211348 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:52.211348 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:52.211348 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:52.211348 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:52.211348 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:52.211348 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:52.211348 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:52.211348 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:52.211348 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:52.211348 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:52.211348 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:52.212254 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:52.212254 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:52.212254 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:52.212254 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:52.212254 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 19:06:52.212254 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 19:06:52.212254 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 19:06:52.212254 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 19:06:52.212254 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 19:06:52.212254 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 19:06:52.212254 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 19:06:52.212254 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:06:52.212254 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 527/1042] 10.222.0.61 () {56 vars in 1096 bytes} [Mon Aug 22 19:06:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 19:06:52.217892 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4225d4-f08b-4bb6-94fd-c0e750f739a2 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:52.243929 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 516/1043] 10.222.0.61 () {58 vars in 1282 bytes} [Mon Aug 22 19:06:51 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 22 19:06:52.250006 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6c75a3-e206-48b7-8a4a-3c6061ad4cb7 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:52.280686 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:52.281762 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:52.333618 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4225d4-f08b-4bb6-94fd-c0e750f739a2 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:52.357239 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6c75a3-e206-48b7-8a4a-3c6061ad4cb7 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:52.371781 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fdd93c-f1d2-4853-bf0a-9974438e39ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:52.372624 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fdd93c-f1d2-4853-bf0a-9974438e39ca None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:52.372738 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fdd93c-f1d2-4853-bf0a-9974438e39ca None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:52.372933 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fdd93c-f1d2-4853-bf0a-9974438e39ca None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:52.374151 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fdd93c-f1d2-4853-bf0a-9974438e39ca None admin] RBAC: Adding query filter params (name=admin) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:52.374455 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fdd93c-f1d2-4853-bf0a-9974438e39ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:52.375946 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fdd93c-f1d2-4853-bf0a-9974438e39ca None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:52.384737 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 528/1044] 10.222.0.61 () {56 vars in 1099 bytes} [Mon Aug 22 19:06:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 19:06:52.390424 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fdd93c-f1d2-4853-bf0a-9974438e39ca None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:52.398851 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfc56cd-b46c-4c65-955c-38066456b917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:52.399986 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfc56cd-b46c-4c65-955c-38066456b917 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:52.400328 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfc56cd-b46c-4c65-955c-38066456b917 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:52.400659 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfc56cd-b46c-4c65-955c-38066456b917 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:52.412612 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfc56cd-b46c-4c65-955c-38066456b917 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:52.414115 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfc56cd-b46c-4c65-955c-38066456b917 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:52.429556 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-adfc56cd-b46c-4c65-955c-38066456b917 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:06:52.429556 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:52.429556 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:52.429556 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:52.429556 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:52.429556 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:52.429556 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:52.429556 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:52.429556 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:52.429556 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:52.429556 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:52.429556 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:52.429556 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 19:06:52.429556 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 19:06:52.429556 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 19:06:52.429556 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 19:06:52.431057 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:52.431057 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:52.431057 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:52.431057 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:52.431057 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:52.431057 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:52.431057 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:52.431057 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:52.431057 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:52.431057 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:52.431057 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:52.431057 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:52.431057 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:52.431057 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:52.431057 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:52.431057 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:52.431057 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 19:06:52.432843 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 19:06:52.432843 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:06:52.432843 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:06:52.432843 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 517/1045] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:06:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 19:06:52.437053 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfc56cd-b46c-4c65-955c-38066456b917 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:52.497134 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fdd93c-f1d2-4853-bf0a-9974438e39ca None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:52.515729 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:52.534791 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:52.536030 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbdd44f-742e-4330-ad1a-3fe6a133b87b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:52.536667 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbdd44f-742e-4330-ad1a-3fe6a133b87b None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:52.537031 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbdd44f-742e-4330-ad1a-3fe6a133b87b None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:52.537213 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbdd44f-742e-4330-ad1a-3fe6a133b87b None admin] PATH_INFO: `/v3/users/6484eff5ce3446a0b51018a04fc19b35` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:52.551699 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbdd44f-742e-4330-ad1a-3fe6a133b87b None admin] RBAC: Authorizing `identity:get_user(user_id=6484eff5ce3446a0b51018a04fc19b35)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:52.551834 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfc56cd-b46c-4c65-955c-38066456b917 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:52.553136 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbdd44f-742e-4330-ad1a-3fe6a133b87b None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:52.555064 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 529/1046] 10.222.0.61 () {56 vars in 1177 bytes} [Mon Aug 22 19:06:52 2022] GET /identity/v3/users/6484eff5ce3446a0b51018a04fc19b35 => generated 285 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 19:06:52.560525 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbdd44f-742e-4330-ad1a-3fe6a133b87b None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:52.589901 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ca8f49-1adb-4c76-82b4-4a87873a95db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:52.590505 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ca8f49-1adb-4c76-82b4-4a87873a95db None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:52.590775 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ca8f49-1adb-4c76-82b4-4a87873a95db None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:52.591045 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ca8f49-1adb-4c76-82b4-4a87873a95db None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:52.591734 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ca8f49-1adb-4c76-82b4-4a87873a95db None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:52.592174 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ca8f49-1adb-4c76-82b4-4a87873a95db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:52.593626 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ca8f49-1adb-4c76-82b4-4a87873a95db None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:52.611165 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 518/1047] 10.222.0.61 () {56 vars in 1109 bytes} [Mon Aug 22 19:06:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:52.612697 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ca8f49-1adb-4c76-82b4-4a87873a95db None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:52.662760 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbdd44f-742e-4330-ad1a-3fe6a133b87b None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:52.701147 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89db0a9a-5b6b-44f2-a26b-0476860c56c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:52.701822 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89db0a9a-5b6b-44f2-a26b-0476860c56c8 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:52.702107 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89db0a9a-5b6b-44f2-a26b-0476860c56c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:52.702382 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89db0a9a-5b6b-44f2-a26b-0476860c56c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:52.720572 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ca8f49-1adb-4c76-82b4-4a87873a95db None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:52.746058 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:52.758179 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68669ae-11c1-4d93-8211-2d640d8637a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:52.759077 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68669ae-11c1-4d93-8211-2d640d8637a5 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:52.759538 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68669ae-11c1-4d93-8211-2d640d8637a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:52.759975 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68669ae-11c1-4d93-8211-2d640d8637a5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:52.767022 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68669ae-11c1-4d93-8211-2d640d8637a5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:52.768466 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68669ae-11c1-4d93-8211-2d640d8637a5 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:52.773344 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89db0a9a-5b6b-44f2-a26b-0476860c56c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:52.774784 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89db0a9a-5b6b-44f2-a26b-0476860c56c8 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:52.779364 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-c68669ae-11c1-4d93-8211-2d640d8637a5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 19:06:52.779364 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:52.779364 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:52.779364 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:52.779364 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:52.779364 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:52.779364 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:52.779364 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:52.779364 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:52.779364 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:52.779364 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:52.779364 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:52.779364 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 19:06:52.779364 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 19:06:52.779364 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 19:06:52.780910 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 19:06:52.780910 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:52.780910 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:52.780910 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:52.780910 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:52.780910 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:52.780910 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:52.780910 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:52.780910 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:52.780910 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:52.780910 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:52.780910 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:52.780910 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:52.780910 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:52.780910 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:52.780910 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:52.780910 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:52.787030 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 19:06:52.787030 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 19:06:52.787030 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 19:06:52.787030 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 19:06:52.787030 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 19:06:52.787030 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 19:06:52.787030 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 19:06:52.787030 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:06:52.787030 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 519/1048] 10.222.0.61 () {56 vars in 1111 bytes} [Mon Aug 22 19:06:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 19:06:52.787030 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68669ae-11c1-4d93-8211-2d640d8637a5 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:52.790352 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:52.883654 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 530/1049] 10.222.0.61 () {58 vars in 1282 bytes} [Mon Aug 22 19:06:52 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 22 19:06:52.884937 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77350fcd-5075-4f40-9e58-7e68941a61d6 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:52.885218 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77350fcd-5075-4f40-9e58-7e68941a61d6 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:52.885488 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77350fcd-5075-4f40-9e58-7e68941a61d6 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:52.886874 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 531/1050] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:06:52 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:06:52.889482 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77350fcd-5075-4f40-9e58-7e68941a61d6 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:52.891879 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68669ae-11c1-4d93-8211-2d640d8637a5 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:52.926243 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a471b29b-6f29-4417-aad8-317ea26d0999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:52.926855 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a471b29b-6f29-4417-aad8-317ea26d0999 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:52.927136 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a471b29b-6f29-4417-aad8-317ea26d0999 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:52.927412 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a471b29b-6f29-4417-aad8-317ea26d0999 None admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:52.928134 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a471b29b-6f29-4417-aad8-317ea26d0999 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:52.928441 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a471b29b-6f29-4417-aad8-317ea26d0999 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:52.929867 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a471b29b-6f29-4417-aad8-317ea26d0999 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:52.941467 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 520/1051] 10.222.0.61 () {56 vars in 1148 bytes} [Mon Aug 22 19:06:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 19:06:52.942608 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec22e5fe-8710-471a-ae84-7d909e165a99 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:52.942893 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec22e5fe-8710-471a-ae84-7d909e165a99 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:52.943169 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec22e5fe-8710-471a-ae84-7d909e165a99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:52.960219 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-ec22e5fe-8710-471a-ae84-7d909e165a99 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:06:52.961500 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-ec22e5fe-8710-471a-ae84-7d909e165a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:06:52.971251 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:52.998337 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77350fcd-5075-4f40-9e58-7e68941a61d6 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:53.019466 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:53.027959 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 521/1052] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:06:52 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 19:06:53.028750 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec22e5fe-8710-471a-ae84-7d909e165a99 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:53.044476 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b52536a-6f2a-4e14-9a49-e5f90688594e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:53.045143 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b52536a-6f2a-4e14-9a49-e5f90688594e None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:53.046016 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b52536a-6f2a-4e14-9a49-e5f90688594e None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:53.046290 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b52536a-6f2a-4e14-9a49-e5f90688594e None admin] PATH_INFO: `/v3/projects/9d2c1e4458e84033a49b2bee6c1af26e` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:53.047412 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b52536a-6f2a-4e14-9a49-e5f90688594e None admin] RBAC: Authorizing `identity:get_project(project_id=9d2c1e4458e84033a49b2bee6c1af26e)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:53.048872 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b52536a-6f2a-4e14-9a49-e5f90688594e None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:53.050635 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 532/1053] 10.222.0.61 () {56 vars in 1186 bytes} [Mon Aug 22 19:06:52 2022] GET /identity/v3/projects/9d2c1e4458e84033a49b2bee6c1af26e => generated 311 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:53.056341 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b52536a-6f2a-4e14-9a49-e5f90688594e None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:53.137561 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec22e5fe-8710-471a-ae84-7d909e165a99 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:53.169485 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b52536a-6f2a-4e14-9a49-e5f90688594e None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:53.177382 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d453a0-c0a9-4041-b486-2976e67b8ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:53.178183 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d453a0-c0a9-4041-b486-2976e67b8ca0 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:53.178728 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d453a0-c0a9-4041-b486-2976e67b8ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:53.179002 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d453a0-c0a9-4041-b486-2976e67b8ca0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:53.179759 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d453a0-c0a9-4041-b486-2976e67b8ca0 None admin] RBAC: Adding query filter params (role.id=aa9eb04acf884c51ad2adb05380bc7a1, scope.project.id=e07dc5ebdc504c6da706a2d21c0a3970, user.id=e7f43e2756b746a6825ca1a06b7d7c91) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:53.180182 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d453a0-c0a9-4041-b486-2976e67b8ca0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:53.181636 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d453a0-c0a9-4041-b486-2976e67b8ca0 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:53.195674 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 522/1054] 10.222.0.61 () {56 vars in 1374 bytes} [Mon Aug 22 19:06:53 2022] GET /identity/v3/role_assignments?user.id=e7f43e2756b746a6825ca1a06b7d7c91&scope.project.id=e07dc5ebdc504c6da706a2d21c0a3970&role.id=aa9eb04acf884c51ad2adb05380bc7a1 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:53.213326 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca53fbe-a29f-4b05-b6fb-70f72d04e238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:53.214059 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca53fbe-a29f-4b05-b6fb-70f72d04e238 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:53.214441 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca53fbe-a29f-4b05-b6fb-70f72d04e238 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:53.214788 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca53fbe-a29f-4b05-b6fb-70f72d04e238 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:53.215618 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca53fbe-a29f-4b05-b6fb-70f72d04e238 None admin] RBAC: Adding query filter params (role.id=163c30c3b3c246beb849ce176f02d9da, scope.project.id=9d2c1e4458e84033a49b2bee6c1af26e, user.id=6484eff5ce3446a0b51018a04fc19b35) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:53.216032 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca53fbe-a29f-4b05-b6fb-70f72d04e238 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:53.218163 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca53fbe-a29f-4b05-b6fb-70f72d04e238 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:53.224932 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 533/1055] 10.222.0.61 () {56 vars in 1374 bytes} [Mon Aug 22 19:06:53 2022] GET /identity/v3/role_assignments?user.id=6484eff5ce3446a0b51018a04fc19b35&scope.project.id=9d2c1e4458e84033a49b2bee6c1af26e&role.id=163c30c3b3c246beb849ce176f02d9da => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 19:06:53.302809 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d453a0-c0a9-4041-b486-2976e67b8ca0 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:53.375474 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:53.426336 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d453a0-c0a9-4041-b486-2976e67b8ca0 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:53.461259 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fd497a-4391-4fe5-b2b6-91653b8f94e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:53.461906 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fd497a-4391-4fe5-b2b6-91653b8f94e4 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:53.462163 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fd497a-4391-4fe5-b2b6-91653b8f94e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:53.462429 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fd497a-4391-4fe5-b2b6-91653b8f94e4 None admin] PATH_INFO: `/v3/services/image` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:53.463013 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fd497a-4391-4fe5-b2b6-91653b8f94e4 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:53.464395 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fd497a-4391-4fe5-b2b6-91653b8f94e4 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:53.473749 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-93fd497a-4391-4fe5-b2b6-91653b8f94e4 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 22 19:06:53.473749 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:53.473749 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 19:06:53.473749 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 19:06:53.473749 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 19:06:53.473749 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 19:06:53.473749 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 19:06:53.473749 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 19:06:53.473749 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 22 19:06:53.473749 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:06:53.473749 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 19:06:53.473749 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:06:53.473749 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:53.473749 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:53.473749 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:53.473749 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:53.473749 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:53.474758 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:53.474758 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:53.474758 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:53.474758 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:53.474758 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:53.474758 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:53.474758 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 19:06:53.474758 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 19:06:53.474758 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 19:06:53.474758 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 19:06:53.474758 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:53.474758 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:53.474758 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:53.474758 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:53.474758 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:53.474758 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:53.475632 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:53.475632 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:53.475632 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:53.475632 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:53.475632 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:53.475632 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:53.475632 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:53.475632 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:53.475632 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:53.475632 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:53.475632 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 19:06:53.475632 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 19:06:53.475632 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 22 19:06:53.475632 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:06:53.475632 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 523/1056] 10.222.0.61 () {56 vars in 1105 bytes} [Mon Aug 22 19:06:53 2022] GET /identity/v3/services/image => generated 86 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:53.481514 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca53fbe-a29f-4b05-b6fb-70f72d04e238 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:53.491283 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:53.587321 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca53fbe-a29f-4b05-b6fb-70f72d04e238 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:53.622707 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98175f5-13f3-4ca4-b29c-b3de3bebf1e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:53.623506 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98175f5-13f3-4ca4-b29c-b3de3bebf1e1 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:53.623912 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98175f5-13f3-4ca4-b29c-b3de3bebf1e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:53.624290 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98175f5-13f3-4ca4-b29c-b3de3bebf1e1 None admin] PATH_INFO: `/v3/services` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:53.625023 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98175f5-13f3-4ca4-b29c-b3de3bebf1e1 None admin] RBAC: Adding query filter params (name=image) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:53.625432 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98175f5-13f3-4ca4-b29c-b3de3bebf1e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:53.626974 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98175f5-13f3-4ca4-b29c-b3de3bebf1e1 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:53.632291 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 534/1057] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:06:53 2022] GET /identity/v3/services?name=image => generated 122 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:53.637967 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fd497a-4391-4fe5-b2b6-91653b8f94e4 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:53.740946 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fd497a-4391-4fe5-b2b6-91653b8f94e4 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:53.762636 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:53.775016 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4557b1d3-b61b-43db-be33-f18f2bcd9772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:53.775710 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4557b1d3-b61b-43db-be33-f18f2bcd9772 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:53.776050 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4557b1d3-b61b-43db-be33-f18f2bcd9772 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:53.776384 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4557b1d3-b61b-43db-be33-f18f2bcd9772 None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:53.777074 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4557b1d3-b61b-43db-be33-f18f2bcd9772 None admin] RBAC: Adding query filter params (type=image) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:53.777439 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4557b1d3-b61b-43db-be33-f18f2bcd9772 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:53.778864 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4557b1d3-b61b-43db-be33-f18f2bcd9772 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:53.788992 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 524/1058] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:06:53 2022] GET /identity/v3/services?type=image => generated 350 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:53.789981 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98175f5-13f3-4ca4-b29c-b3de3bebf1e1 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:53.859110 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:53.892029 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98175f5-13f3-4ca4-b29c-b3de3bebf1e1 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:53.924694 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d084f5ca-16c7-44bd-b6c3-c37f247c96c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:53.925364 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d084f5ca-16c7-44bd-b6c3-c37f247c96c4 None admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:53.925720 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d084f5ca-16c7-44bd-b6c3-c37f247c96c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:53.926042 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d084f5ca-16c7-44bd-b6c3-c37f247c96c4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:53.926757 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d084f5ca-16c7-44bd-b6c3-c37f247c96c4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:53.928208 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d084f5ca-16c7-44bd-b6c3-c37f247c96c4 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:53.955667 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 535/1059] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 19:06:53 2022] POST /identity/v3/endpoints => generated 327 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:06:55.103085 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f7dff1-bb7a-4b1c-9b47-a7441453367f None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:55.103628 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f7dff1-bb7a-4b1c-9b47-a7441453367f None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:55.103628 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f7dff1-bb7a-4b1c-9b47-a7441453367f None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:55.107631 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 525/1060] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:06:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:06:55.107773 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70be088-105f-4280-98c8-b31054c3f4a6 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:55.107773 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70be088-105f-4280-98c8-b31054c3f4a6 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:55.107773 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70be088-105f-4280-98c8-b31054c3f4a6 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:55.107773 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 536/1061] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:06:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 19:06:55.111750 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bb7b7d-41e0-4118-b1b9-a15446901815 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:55.111750 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bb7b7d-41e0-4118-b1b9-a15446901815 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:55.111750 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bb7b7d-41e0-4118-b1b9-a15446901815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:55.115743 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c53da6c-98a2-4b3e-8812-c3f375a71057 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:55.116036 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c53da6c-98a2-4b3e-8812-c3f375a71057 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:55.116309 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c53da6c-98a2-4b3e-8812-c3f375a71057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:55.130016 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-75bb7b7d-41e0-4118-b1b9-a15446901815 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:06:55.131217 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-75bb7b7d-41e0-4118-b1b9-a15446901815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:06:55.136069 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-1c53da6c-98a2-4b3e-8812-c3f375a71057 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:06:55.137248 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-1c53da6c-98a2-4b3e-8812-c3f375a71057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:06:55.213897 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 526/1062] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:06:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 22 19:06:55.220713 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 537/1063] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:06:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 19:06:55.476038 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bb7b7d-41e0-4118-b1b9-a15446901815 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:55.502760 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c53da6c-98a2-4b3e-8812-c3f375a71057 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:55.561788 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:55.581227 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bb7b7d-41e0-4118-b1b9-a15446901815 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:55.596350 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:55.613014 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c53da6c-98a2-4b3e-8812-c3f375a71057 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:55.619817 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb87a4f-4ac9-4244-bf5d-1d4f5015fbb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:55.620460 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb87a4f-4ac9-4244-bf5d-1d4f5015fbb7 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:55.620749 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb87a4f-4ac9-4244-bf5d-1d4f5015fbb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:55.621009 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb87a4f-4ac9-4244-bf5d-1d4f5015fbb7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:55.632106 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb87a4f-4ac9-4244-bf5d-1d4f5015fbb7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 19:06:55.636868 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-5cb87a4f-4ac9-4244-bf5d-1d4f5015fbb7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 22 19:06:55.636868 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:55.636868 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:55.636868 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:55.636868 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:55.636868 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:55.636868 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:55.636868 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:55.636868 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:55.636868 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:55.636868 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:55.636868 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:55.636868 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 19:06:55.636868 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 19:06:55.636868 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 19:06:55.636868 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 19:06:55.637837 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 19:06:55.637837 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 19:06:55.637837 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 19:06:55.637837 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 19:06:55.637837 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:55.637837 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:55.637837 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:55.637837 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:55.637837 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:55.637837 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:55.637837 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:55.637837 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:55.637837 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:55.637837 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:55.637837 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:55.637837 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:55.638717 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:55.638717 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:55.638717 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:55.638717 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:55.638717 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 19:06:55.638717 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 19:06:55.638717 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 19:06:55.638717 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 19:06:55.638717 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 19:06:55.638717 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 19:06:55.638717 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 22 19:06:55.638717 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:06:55.638717 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 527/1064] 10.222.0.61 () {56 vars in 1120 bytes} [Mon Aug 22 19:06:55 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:55.644097 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb87a4f-4ac9-4244-bf5d-1d4f5015fbb7 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:55.652890 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1498edaa-4339-4056-8197-a50f6e190908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:55.653532 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1498edaa-4339-4056-8197-a50f6e190908 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:55.653810 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1498edaa-4339-4056-8197-a50f6e190908 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:55.654082 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1498edaa-4339-4056-8197-a50f6e190908 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:55.666290 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1498edaa-4339-4056-8197-a50f6e190908 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 19:06:55.671569 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-1498edaa-4339-4056-8197-a50f6e190908 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 19:06:55.671569 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:55.671569 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:55.671569 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:55.671569 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:55.671569 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:55.671569 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:55.671569 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:55.671569 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:55.671569 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:55.671569 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:55.671569 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:55.671569 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 19:06:55.671569 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 19:06:55.671569 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 19:06:55.671569 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 19:06:55.672780 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 19:06:55.672780 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 19:06:55.672780 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 19:06:55.672780 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 19:06:55.672780 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:55.672780 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:55.672780 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:55.672780 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:55.672780 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:55.672780 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:55.672780 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:55.672780 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:55.672780 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:55.672780 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:55.672780 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:55.672780 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:55.673844 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:55.673844 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:55.673844 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:55.673844 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:55.673844 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 19:06:55.673844 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 19:06:55.673844 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 19:06:55.673844 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 19:06:55.673844 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 19:06:55.673844 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 19:06:55.673844 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 19:06:55.673844 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:06:55.675952 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 538/1065] 10.222.0.61 () {56 vars in 1096 bytes} [Mon Aug 22 19:06:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 19:06:55.678598 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1498edaa-4339-4056-8197-a50f6e190908 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:55.759963 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb87a4f-4ac9-4244-bf5d-1d4f5015fbb7 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:55.782170 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1498edaa-4339-4056-8197-a50f6e190908 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:55.797182 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c52527-9251-4cc4-a15e-07634d5b2be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:55.797794 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c52527-9251-4cc4-a15e-07634d5b2be3 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:55.798077 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c52527-9251-4cc4-a15e-07634d5b2be3 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:55.798352 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c52527-9251-4cc4-a15e-07634d5b2be3 None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:55.799021 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c52527-9251-4cc4-a15e-07634d5b2be3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:55.799324 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c52527-9251-4cc4-a15e-07634d5b2be3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:55.800153 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:55.800725 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c52527-9251-4cc4-a15e-07634d5b2be3 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:55.809894 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 528/1066] 10.222.0.61 () {56 vars in 1115 bytes} [Mon Aug 22 19:06:55 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 19:06:55.815379 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c52527-9251-4cc4-a15e-07634d5b2be3 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:55.821209 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e982e151-e8e9-4a97-a639-69da2265a665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:55.821832 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e982e151-e8e9-4a97-a639-69da2265a665 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:55.822113 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e982e151-e8e9-4a97-a639-69da2265a665 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:55.822388 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e982e151-e8e9-4a97-a639-69da2265a665 None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:55.823063 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e982e151-e8e9-4a97-a639-69da2265a665 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:55.823370 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e982e151-e8e9-4a97-a639-69da2265a665 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:55.824779 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e982e151-e8e9-4a97-a639-69da2265a665 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:55.835694 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 539/1067] 10.222.0.61 () {56 vars in 1099 bytes} [Mon Aug 22 19:06:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 19:06:55.839143 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e982e151-e8e9-4a97-a639-69da2265a665 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:55.856444 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:55.923268 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c52527-9251-4cc4-a15e-07634d5b2be3 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:55.940186 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e982e151-e8e9-4a97-a639-69da2265a665 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:55.957121 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2659f5a-8965-4dfb-8f24-60965e6be3c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:55.957740 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2659f5a-8965-4dfb-8f24-60965e6be3c6 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:55.958017 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2659f5a-8965-4dfb-8f24-60965e6be3c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:55.958281 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2659f5a-8965-4dfb-8f24-60965e6be3c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:55.968590 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2659f5a-8965-4dfb-8f24-60965e6be3c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:55.970009 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2659f5a-8965-4dfb-8f24-60965e6be3c6 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:55.974311 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822dd3d6-5df4-4145-8378-1f08ec20cc7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:55.974939 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822dd3d6-5df4-4145-8378-1f08ec20cc7d None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:55.975218 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822dd3d6-5df4-4145-8378-1f08ec20cc7d None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:55.975490 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822dd3d6-5df4-4145-8378-1f08ec20cc7d None admin] PATH_INFO: `/v3/users/6484eff5ce3446a0b51018a04fc19b35` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:55.976668 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822dd3d6-5df4-4145-8378-1f08ec20cc7d None admin] RBAC: Authorizing `identity:get_user(user_id=6484eff5ce3446a0b51018a04fc19b35)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:55.978083 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822dd3d6-5df4-4145-8378-1f08ec20cc7d None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:55.984425 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 540/1068] 10.222.0.61 () {56 vars in 1177 bytes} [Mon Aug 22 19:06:55 2022] GET /identity/v3/users/6484eff5ce3446a0b51018a04fc19b35 => generated 285 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 19:06:55.984425 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a1a96a-3e4f-460a-bfae-35eecc54dd49 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:55.984425 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a1a96a-3e4f-460a-bfae-35eecc54dd49 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:55.984425 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a1a96a-3e4f-460a-bfae-35eecc54dd49 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:55.984425 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 541/1069] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:06:55 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:06:55.984812 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-f2659f5a-8965-4dfb-8f24-60965e6be3c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:06:55.984812 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:55.984812 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:55.984812 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:55.984812 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:55.984812 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:55.984812 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:55.984812 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:55.984812 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:55.984812 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:55.984812 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:55.984812 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:55.984812 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 19:06:55.984812 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 19:06:55.984812 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 19:06:55.984812 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 19:06:55.985775 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:55.985775 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:55.985775 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:55.985775 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:55.985775 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:55.985775 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:55.985775 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:55.985775 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:55.985775 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:55.985775 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:55.985775 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:55.985775 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:55.985775 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:55.985775 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:55.985775 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:55.985775 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:55.985775 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 19:06:55.986716 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 19:06:55.986716 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:06:55.986716 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:06:55.986901 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a1a96a-3e4f-460a-bfae-35eecc54dd49 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:55.986973 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 529/1070] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:06:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 19:06:55.988208 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cc0cbb-f62c-4091-b65d-3e0dbd8caa70 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:55.988738 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cc0cbb-f62c-4091-b65d-3e0dbd8caa70 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:55.989020 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cc0cbb-f62c-4091-b65d-3e0dbd8caa70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:56.007372 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-d5cc0cbb-f62c-4091-b65d-3e0dbd8caa70 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:06:56.008627 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-d5cc0cbb-f62c-4091-b65d-3e0dbd8caa70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:06:56.036775 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:56.075940 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 530/1071] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:06:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 22 19:06:56.083729 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cc0cbb-f62c-4091-b65d-3e0dbd8caa70 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:56.096850 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:56.099633 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a1a96a-3e4f-460a-bfae-35eecc54dd49 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:56.138672 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765ff7f1-39db-45c6-8f7d-42895b4adbfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:56.139481 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765ff7f1-39db-45c6-8f7d-42895b4adbfe None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:56.139720 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765ff7f1-39db-45c6-8f7d-42895b4adbfe None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:56.139975 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765ff7f1-39db-45c6-8f7d-42895b4adbfe None admin] PATH_INFO: `/v3/projects/9d2c1e4458e84033a49b2bee6c1af26e` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:56.141822 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765ff7f1-39db-45c6-8f7d-42895b4adbfe None admin] RBAC: Authorizing `identity:get_project(project_id=9d2c1e4458e84033a49b2bee6c1af26e)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:56.146739 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765ff7f1-39db-45c6-8f7d-42895b4adbfe None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:56.148502 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 542/1072] 10.222.0.61 () {56 vars in 1186 bytes} [Mon Aug 22 19:06:55 2022] GET /identity/v3/projects/9d2c1e4458e84033a49b2bee6c1af26e => generated 311 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:56.154179 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765ff7f1-39db-45c6-8f7d-42895b4adbfe None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:56.196881 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cc0cbb-f62c-4091-b65d-3e0dbd8caa70 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:56.234224 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269b2a9a-eef1-46b8-819c-f223748e8604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:56.234831 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269b2a9a-eef1-46b8-819c-f223748e8604 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:56.235111 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269b2a9a-eef1-46b8-819c-f223748e8604 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:56.235382 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269b2a9a-eef1-46b8-819c-f223748e8604 None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:56.236059 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269b2a9a-eef1-46b8-819c-f223748e8604 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:56.236361 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269b2a9a-eef1-46b8-819c-f223748e8604 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:56.237720 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269b2a9a-eef1-46b8-819c-f223748e8604 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:56.254390 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 531/1073] 10.222.0.61 () {56 vars in 1109 bytes} [Mon Aug 22 19:06:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:56.256142 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269b2a9a-eef1-46b8-819c-f223748e8604 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:56.261620 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765ff7f1-39db-45c6-8f7d-42895b4adbfe None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:56.275916 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:56.301491 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27af3aa6-8e8e-4a50-925b-3dea674c5d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:56.302082 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27af3aa6-8e8e-4a50-925b-3dea674c5d7c None admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:56.302366 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27af3aa6-8e8e-4a50-925b-3dea674c5d7c None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:56.303045 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27af3aa6-8e8e-4a50-925b-3dea674c5d7c None admin] PATH_INFO: `/v3/projects/9d2c1e4458e84033a49b2bee6c1af26e/users/6484eff5ce3446a0b51018a04fc19b35/roles/163c30c3b3c246beb849ce176f02d9da` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:56.305379 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27af3aa6-8e8e-4a50-925b-3dea674c5d7c None admin] RBAC: Authorizing `identity:create_grant(project_id=9d2c1e4458e84033a49b2bee6c1af26e, user_id=6484eff5ce3446a0b51018a04fc19b35, role_id=163c30c3b3c246beb849ce176f02d9da)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:56.307164 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27af3aa6-8e8e-4a50-925b-3dea674c5d7c None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:56.328768 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 543/1074] 10.222.0.61 () {58 vars in 1439 bytes} [Mon Aug 22 19:06:56 2022] PUT /identity/v3/projects/9d2c1e4458e84033a49b2bee6c1af26e/users/6484eff5ce3446a0b51018a04fc19b35/roles/163c30c3b3c246beb849ce176f02d9da => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 19:06:56.341141 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27af3aa6-8e8e-4a50-925b-3dea674c5d7c None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:56.352059 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:56.381410 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269b2a9a-eef1-46b8-819c-f223748e8604 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:56.423255 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadec310-7639-48e1-98bd-55c12b2588d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:56.423983 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadec310-7639-48e1-98bd-55c12b2588d2 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:56.424266 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadec310-7639-48e1-98bd-55c12b2588d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:56.424537 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadec310-7639-48e1-98bd-55c12b2588d2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:56.433652 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadec310-7639-48e1-98bd-55c12b2588d2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:56.435122 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadec310-7639-48e1-98bd-55c12b2588d2 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:56.448957 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-dadec310-7639-48e1-98bd-55c12b2588d2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 22 19:06:56.448957 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:56.448957 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:56.448957 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:56.448957 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:56.448957 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:56.448957 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:56.448957 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:56.448957 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:56.448957 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:56.448957 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:56.448957 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:56.448957 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 19:06:56.448957 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 19:06:56.448957 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 19:06:56.448957 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 19:06:56.450020 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:56.450020 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:56.450020 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 19:06:56.450020 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 19:06:56.450020 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 19:06:56.450020 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 19:06:56.450020 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:56.450020 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:56.450020 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:56.450020 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:56.450020 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:56.450020 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:56.450020 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:56.450020 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:56.450020 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:56.450020 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:56.450020 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:56.450020 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:56.450983 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:56.450983 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:56.450983 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 19:06:56.450983 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 19:06:56.450983 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 19:06:56.450983 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 19:06:56.450983 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 19:06:56.450983 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 19:06:56.450983 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 22 19:06:56.450983 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:06:56.450983 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 532/1075] 10.222.0.61 () {56 vars in 1093 bytes} [Mon Aug 22 19:06:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 19:06:56.456355 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadec310-7639-48e1-98bd-55c12b2588d2 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:56.458737 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27af3aa6-8e8e-4a50-925b-3dea674c5d7c None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:56.500242 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07427e88-49bd-47cd-85c8-f629960c5e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:56.500855 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07427e88-49bd-47cd-85c8-f629960c5e7f None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:56.501134 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07427e88-49bd-47cd-85c8-f629960c5e7f None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:56.501404 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07427e88-49bd-47cd-85c8-f629960c5e7f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:56.544932 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:56.572570 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadec310-7639-48e1-98bd-55c12b2588d2 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:56.574132 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07427e88-49bd-47cd-85c8-f629960c5e7f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:56.575677 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07427e88-49bd-47cd-85c8-f629960c5e7f None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:56.608765 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497d3d55-3a29-42bc-be71-fea04958500a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:56.609372 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d3d55-3a29-42bc-be71-fea04958500a None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:56.609645 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d3d55-3a29-42bc-be71-fea04958500a None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:56.609914 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d3d55-3a29-42bc-be71-fea04958500a None admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:56.610639 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497d3d55-3a29-42bc-be71-fea04958500a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:56.610938 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497d3d55-3a29-42bc-be71-fea04958500a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:56.612687 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497d3d55-3a29-42bc-be71-fea04958500a None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:56.616122 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:56.637609 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 533/1076] 10.222.0.61 () {56 vars in 1133 bytes} [Mon Aug 22 19:06:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 19:06:56.638924 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497d3d55-3a29-42bc-be71-fea04958500a None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:56.687349 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 544/1077] 10.222.0.61 () {58 vars in 1282 bytes} [Mon Aug 22 19:06:56 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 22 19:06:56.693151 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07427e88-49bd-47cd-85c8-f629960c5e7f None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:56.749708 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497d3d55-3a29-42bc-be71-fea04958500a None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:56.776223 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:56.788386 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51ff545-dd0f-459e-b3b5-7bc6cef267b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:56.789024 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51ff545-dd0f-459e-b3b5-7bc6cef267b0 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:56.789302 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51ff545-dd0f-459e-b3b5-7bc6cef267b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:56.789620 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51ff545-dd0f-459e-b3b5-7bc6cef267b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:56.800511 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51ff545-dd0f-459e-b3b5-7bc6cef267b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:56.801944 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51ff545-dd0f-459e-b3b5-7bc6cef267b0 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:56.805257 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07427e88-49bd-47cd-85c8-f629960c5e7f None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:56.816346 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-a51ff545-dd0f-459e-b3b5-7bc6cef267b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:06:56.816346 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:56.816346 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:56.816346 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:56.816346 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:56.816346 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:56.816346 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:56.816346 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:56.816346 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:56.816346 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:56.816346 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:56.816346 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:56.816346 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 19:06:56.816346 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 19:06:56.816346 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 19:06:56.816346 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 19:06:56.817305 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:56.817305 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:56.817305 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:56.817305 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:56.817305 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:56.817305 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:56.817305 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:56.817305 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:56.817305 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:56.817305 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:56.817305 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:56.817305 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:56.817305 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:56.817305 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:56.817305 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:56.817305 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:56.817305 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 19:06:56.818288 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 19:06:56.818288 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:06:56.818288 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:06:56.818288 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 534/1078] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:06:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 19:06:56.823542 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51ff545-dd0f-459e-b3b5-7bc6cef267b0 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:56.844528 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a14af7a-de07-48ad-b461-822bbfcf90e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:56.845161 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a14af7a-de07-48ad-b461-822bbfcf90e2 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:56.845442 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a14af7a-de07-48ad-b461-822bbfcf90e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:56.845714 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a14af7a-de07-48ad-b461-822bbfcf90e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:56.855953 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a14af7a-de07-48ad-b461-822bbfcf90e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:56.857393 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a14af7a-de07-48ad-b461-822bbfcf90e2 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:56.872644 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:56.876826 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-0a14af7a-de07-48ad-b461-822bbfcf90e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:06:56.876826 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:56.876826 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:56.876826 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:56.876826 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:56.876826 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:56.876826 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:56.876826 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:56.876826 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:56.876826 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:56.876826 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:56.876826 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:56.876826 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 19:06:56.876826 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 19:06:56.876826 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 19:06:56.876826 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 19:06:56.877798 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:56.877798 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:56.877798 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:56.877798 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:56.877798 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:56.877798 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:56.877798 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:56.877798 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:56.877798 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:56.877798 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:56.877798 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:56.877798 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:56.877798 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:56.877798 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:56.877798 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:56.877798 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:56.877798 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 19:06:56.878686 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 19:06:56.878686 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:06:56.878686 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:06:56.878686 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 545/1079] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:06:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 19:06:56.883861 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a14af7a-de07-48ad-b461-822bbfcf90e2 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:56.930122 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51ff545-dd0f-459e-b3b5-7bc6cef267b0 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:56.964264 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84d4bee-03c1-4016-aba8-4437320c100b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:56.964876 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84d4bee-03c1-4016-aba8-4437320c100b None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:56.965156 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84d4bee-03c1-4016-aba8-4437320c100b None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:56.965426 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84d4bee-03c1-4016-aba8-4437320c100b None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:56.966076 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84d4bee-03c1-4016-aba8-4437320c100b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:56.966378 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84d4bee-03c1-4016-aba8-4437320c100b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:56.968315 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84d4bee-03c1-4016-aba8-4437320c100b None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:56.980136 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 535/1080] 10.222.0.61 () {56 vars in 1109 bytes} [Mon Aug 22 19:06:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 19:06:56.985596 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84d4bee-03c1-4016-aba8-4437320c100b None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:56.990385 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a14af7a-de07-48ad-b461-822bbfcf90e2 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:57.018537 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:57.028409 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172783eb-beb1-4add-acf6-2b054441c01b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:57.029073 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172783eb-beb1-4add-acf6-2b054441c01b None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:57.029359 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172783eb-beb1-4add-acf6-2b054441c01b None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:57.030186 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172783eb-beb1-4add-acf6-2b054441c01b None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:57.030863 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172783eb-beb1-4add-acf6-2b054441c01b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:57.031166 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172783eb-beb1-4add-acf6-2b054441c01b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:57.032577 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172783eb-beb1-4add-acf6-2b054441c01b None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:57.045276 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 546/1081] 10.222.0.61 () {56 vars in 1109 bytes} [Mon Aug 22 19:06:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:57.105291 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84d4bee-03c1-4016-aba8-4437320c100b None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:57.114309 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:57.145236 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6fc682-6ab7-488a-93d6-7063ee43a2d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:57.145831 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6fc682-6ab7-488a-93d6-7063ee43a2d7 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:57.146111 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6fc682-6ab7-488a-93d6-7063ee43a2d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:57.146383 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6fc682-6ab7-488a-93d6-7063ee43a2d7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:57.153655 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6fc682-6ab7-488a-93d6-7063ee43a2d7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:57.155756 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6fc682-6ab7-488a-93d6-7063ee43a2d7 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:57.168030 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-2f6fc682-6ab7-488a-93d6-7063ee43a2d7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 19:06:57.168030 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:57.168030 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:57.168030 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:57.168030 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:57.168030 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:57.168030 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:57.168030 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:57.168030 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:57.168030 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:57.168030 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:57.168030 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:57.168030 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 19:06:57.168030 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 19:06:57.168030 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 19:06:57.169105 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 19:06:57.169105 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:57.169105 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:57.169105 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:57.169105 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:57.169105 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:57.169105 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:57.169105 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:57.169105 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:57.169105 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:57.169105 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:57.169105 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:57.169105 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:57.169105 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:57.169105 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:57.169105 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:57.169105 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:57.175873 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 19:06:57.175873 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 19:06:57.175873 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 19:06:57.175873 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 19:06:57.175873 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 19:06:57.175873 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 19:06:57.175873 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 19:06:57.175873 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:06:57.175873 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 536/1082] 10.222.0.61 () {56 vars in 1111 bytes} [Mon Aug 22 19:06:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 19:06:57.178051 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172783eb-beb1-4add-acf6-2b054441c01b None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:57.285953 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172783eb-beb1-4add-acf6-2b054441c01b None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:57.322473 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568f7a50-2cb4-4973-bd03-5d7697ec6cfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:57.323090 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568f7a50-2cb4-4973-bd03-5d7697ec6cfe None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:57.323365 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568f7a50-2cb4-4973-bd03-5d7697ec6cfe None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:57.323630 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568f7a50-2cb4-4973-bd03-5d7697ec6cfe None admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:57.324379 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568f7a50-2cb4-4973-bd03-5d7697ec6cfe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:57.324676 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568f7a50-2cb4-4973-bd03-5d7697ec6cfe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:57.326128 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568f7a50-2cb4-4973-bd03-5d7697ec6cfe None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:57.338424 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 547/1083] 10.222.0.61 () {56 vars in 1148 bytes} [Mon Aug 22 19:06:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:57.345159 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6fc682-6ab7-488a-93d6-7063ee43a2d7 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:57.449557 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6fc682-6ab7-488a-93d6-7063ee43a2d7 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:57.481428 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b324e361-3b0b-4a23-b673-67ebf172ecc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:57.482071 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b324e361-3b0b-4a23-b673-67ebf172ecc2 None admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:57.482344 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b324e361-3b0b-4a23-b673-67ebf172ecc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:57.483219 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b324e361-3b0b-4a23-b673-67ebf172ecc2 None admin] PATH_INFO: `/v3/projects/e07dc5ebdc504c6da706a2d21c0a3970/users/e7f43e2756b746a6825ca1a06b7d7c91/roles/aa9eb04acf884c51ad2adb05380bc7a1` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:57.485336 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b324e361-3b0b-4a23-b673-67ebf172ecc2 None admin] RBAC: Authorizing `identity:create_grant(project_id=e07dc5ebdc504c6da706a2d21c0a3970, user_id=e7f43e2756b746a6825ca1a06b7d7c91, role_id=aa9eb04acf884c51ad2adb05380bc7a1)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:57.486748 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b324e361-3b0b-4a23-b673-67ebf172ecc2 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:57.499582 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 537/1084] 10.222.0.61 () {58 vars in 1439 bytes} [Mon Aug 22 19:06:57 2022] PUT /identity/v3/projects/e07dc5ebdc504c6da706a2d21c0a3970/users/e7f43e2756b746a6825ca1a06b7d7c91/roles/aa9eb04acf884c51ad2adb05380bc7a1 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:06:58.235019 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb8245b-f62e-4722-8ddb-cc3b869088fe None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:58.235534 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb8245b-f62e-4722-8ddb-cc3b869088fe None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:58.235621 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb8245b-f62e-4722-8ddb-cc3b869088fe None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:58.236939 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 548/1085] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:06:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:06:58.242585 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc3ad08-ab51-4dec-b828-7a61aa260942 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:58.242876 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc3ad08-ab51-4dec-b828-7a61aa260942 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:58.243155 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc3ad08-ab51-4dec-b828-7a61aa260942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:58.262176 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-abc3ad08-ab51-4dec-b828-7a61aa260942 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:06:58.263374 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-abc3ad08-ab51-4dec-b828-7a61aa260942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:06:58.283828 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:58.332769 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 538/1086] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:06:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 19:06:58.604096 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb8245b-f62e-4722-8ddb-cc3b869088fe None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:58.651693 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:58.705259 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb8245b-f62e-4722-8ddb-cc3b869088fe None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:58.737873 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8373d567-8837-41bc-b4fd-c4f5ed6d6138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:58.738748 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8373d567-8837-41bc-b4fd-c4f5ed6d6138 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:58.739028 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8373d567-8837-41bc-b4fd-c4f5ed6d6138 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:58.739300 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8373d567-8837-41bc-b4fd-c4f5ed6d6138 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:58.749513 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8373d567-8837-41bc-b4fd-c4f5ed6d6138 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 19:06:58.754185 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-8373d567-8837-41bc-b4fd-c4f5ed6d6138 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 19:06:58.754185 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:58.754185 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:58.754185 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:58.754185 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:58.754185 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:58.754185 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:58.754185 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:58.754185 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:58.754185 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:58.754185 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:58.754185 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:58.754185 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 19:06:58.754185 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 19:06:58.754185 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 19:06:58.754185 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 19:06:58.755118 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 19:06:58.755118 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 19:06:58.755118 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 19:06:58.755118 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 19:06:58.755118 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:58.755118 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:58.755118 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:58.755118 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:58.755118 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:58.755118 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:58.755118 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:58.755118 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:58.755118 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:58.755118 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:58.755118 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:58.755118 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:58.756041 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:58.756041 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:58.756041 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:58.756041 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:58.756041 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 19:06:58.756041 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 19:06:58.756041 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 19:06:58.756041 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 19:06:58.756041 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 19:06:58.756041 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 19:06:58.756041 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 19:06:58.756041 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:06:58.756041 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 549/1087] 10.222.0.61 () {56 vars in 1096 bytes} [Mon Aug 22 19:06:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:58.761174 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc3ad08-ab51-4dec-b828-7a61aa260942 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:58.860498 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc3ad08-ab51-4dec-b828-7a61aa260942 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:58.893903 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1601135-f056-4ff9-8e01-447f8d1d94bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:58.894501 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1601135-f056-4ff9-8e01-447f8d1d94bb None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:58.894777 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1601135-f056-4ff9-8e01-447f8d1d94bb None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:58.895044 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1601135-f056-4ff9-8e01-447f8d1d94bb None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:58.895731 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1601135-f056-4ff9-8e01-447f8d1d94bb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:58.896031 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1601135-f056-4ff9-8e01-447f8d1d94bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:58.897522 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1601135-f056-4ff9-8e01-447f8d1d94bb None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:58.906067 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 539/1088] 10.222.0.61 () {56 vars in 1099 bytes} [Mon Aug 22 19:06:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:58.911922 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8373d567-8837-41bc-b4fd-c4f5ed6d6138 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:59.009175 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8373d567-8837-41bc-b4fd-c4f5ed6d6138 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:59.015986 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:59.042828 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d41a370-abd9-4102-9d59-01d5780be1e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:59.043461 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d41a370-abd9-4102-9d59-01d5780be1e9 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:59.043772 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d41a370-abd9-4102-9d59-01d5780be1e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:59.044044 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d41a370-abd9-4102-9d59-01d5780be1e9 None admin] PATH_INFO: `/v3/users/6484eff5ce3446a0b51018a04fc19b35` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:59.045709 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d41a370-abd9-4102-9d59-01d5780be1e9 None admin] RBAC: Authorizing `identity:get_user(user_id=6484eff5ce3446a0b51018a04fc19b35)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:59.047201 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d41a370-abd9-4102-9d59-01d5780be1e9 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:59.048839 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 550/1089] 10.222.0.61 () {56 vars in 1177 bytes} [Mon Aug 22 19:06:58 2022] GET /identity/v3/users/6484eff5ce3446a0b51018a04fc19b35 => generated 285 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:59.054717 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1601135-f056-4ff9-8e01-447f8d1d94bb None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:59.073723 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:59.156271 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1601135-f056-4ff9-8e01-447f8d1d94bb None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:59.188216 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea5524a-21f1-42a7-bf30-97872051d9d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:59.188811 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea5524a-21f1-42a7-bf30-97872051d9d4 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:59.189087 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea5524a-21f1-42a7-bf30-97872051d9d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:59.189356 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea5524a-21f1-42a7-bf30-97872051d9d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:59.255321 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea5524a-21f1-42a7-bf30-97872051d9d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:59.256793 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea5524a-21f1-42a7-bf30-97872051d9d4 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:59.274184 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:59.348249 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01816862-d441-427b-bbf4-bbfe3fbcd81a None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:59.348848 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01816862-d441-427b-bbf4-bbfe3fbcd81a None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:59.348848 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01816862-d441-427b-bbf4-bbfe3fbcd81a None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:59.350109 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 551/1090] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:06:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 19:06:59.355690 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 540/1091] 10.222.0.61 () {58 vars in 1282 bytes} [Mon Aug 22 19:06:59 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 22 19:06:59.355836 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36d941b-1918-49f2-b652-59d0786a9c76 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:59.355836 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36d941b-1918-49f2-b652-59d0786a9c76 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:59.355836 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36d941b-1918-49f2-b652-59d0786a9c76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:59.359505 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea5524a-21f1-42a7-bf30-97872051d9d4 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:59.374024 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-d36d941b-1918-49f2-b652-59d0786a9c76 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:06:59.375212 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-d36d941b-1918-49f2-b652-59d0786a9c76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:06:59.442146 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 552/1092] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:06:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 19:06:59.456895 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea5524a-21f1-42a7-bf30-97872051d9d4 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:59.473951 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:59.491375 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7cada3-6654-473d-8690-9782f36d60cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:59.492030 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7cada3-6654-473d-8690-9782f36d60cf None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:59.492304 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7cada3-6654-473d-8690-9782f36d60cf None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:59.492570 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7cada3-6654-473d-8690-9782f36d60cf None admin] PATH_INFO: `/v3/projects/9d2c1e4458e84033a49b2bee6c1af26e` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:59.493674 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7cada3-6654-473d-8690-9782f36d60cf None admin] RBAC: Authorizing `identity:get_project(project_id=9d2c1e4458e84033a49b2bee6c1af26e)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:59.495083 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7cada3-6654-473d-8690-9782f36d60cf None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:59.501727 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 541/1093] 10.222.0.61 () {56 vars in 1186 bytes} [Mon Aug 22 19:06:59 2022] GET /identity/v3/projects/9d2c1e4458e84033a49b2bee6c1af26e => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 19:06:59.503015 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36d941b-1918-49f2-b652-59d0786a9c76 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:59.600371 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36d941b-1918-49f2-b652-59d0786a9c76 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:59.602655 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:06:59.634314 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4b753f-3796-4bd3-887a-48a76a0040d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:59.634907 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4b753f-3796-4bd3-887a-48a76a0040d2 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:59.635184 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4b753f-3796-4bd3-887a-48a76a0040d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:59.635449 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4b753f-3796-4bd3-887a-48a76a0040d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:59.636201 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4b753f-3796-4bd3-887a-48a76a0040d2 None admin] RBAC: Adding query filter params (role.id=163c30c3b3c246beb849ce176f02d9da, scope.project.id=9d2c1e4458e84033a49b2bee6c1af26e, user.id=6484eff5ce3446a0b51018a04fc19b35) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:06:59.636505 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4b753f-3796-4bd3-887a-48a76a0040d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:06:59.637909 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4b753f-3796-4bd3-887a-48a76a0040d2 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:06:59.643466 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 553/1094] 10.222.0.61 () {56 vars in 1374 bytes} [Mon Aug 22 19:06:59 2022] GET /identity/v3/role_assignments?user.id=6484eff5ce3446a0b51018a04fc19b35&scope.project.id=9d2c1e4458e84033a49b2bee6c1af26e&role.id=163c30c3b3c246beb849ce176f02d9da => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:06:59.709511 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7cada3-6654-473d-8690-9782f36d60cf None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:59.812173 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7cada3-6654-473d-8690-9782f36d60cf None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:06:59.846509 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4556df2-2ee9-4fb7-88d5-d4595f531425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:06:59.847138 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4556df2-2ee9-4fb7-88d5-d4595f531425 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:06:59.847417 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4556df2-2ee9-4fb7-88d5-d4595f531425 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:06:59.847716 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4556df2-2ee9-4fb7-88d5-d4595f531425 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:06:59.857592 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4556df2-2ee9-4fb7-88d5-d4595f531425 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 19:06:59.862286 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-f4556df2-2ee9-4fb7-88d5-d4595f531425 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 22 19:06:59.862286 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:06:59.862286 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:06:59.862286 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:06:59.862286 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:06:59.862286 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:06:59.862286 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:06:59.862286 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:06:59.862286 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:06:59.862286 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:06:59.862286 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:06:59.862286 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:06:59.862286 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 19:06:59.862286 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 19:06:59.862286 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 19:06:59.862286 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 19:06:59.863241 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 19:06:59.863241 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 19:06:59.863241 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 19:06:59.863241 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 19:06:59.863241 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:06:59.863241 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:06:59.863241 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:06:59.863241 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:06:59.863241 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:06:59.863241 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:06:59.863241 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:06:59.863241 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:06:59.863241 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:06:59.863241 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:06:59.863241 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:06:59.863241 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:06:59.864758 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:06:59.864758 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:06:59.864758 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:06:59.864758 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:06:59.864758 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 19:06:59.864758 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 19:06:59.864758 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 19:06:59.864758 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 19:06:59.864758 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 19:06:59.864758 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 19:06:59.864758 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 22 19:06:59.864758 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:06:59.864758 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 542/1095] 10.222.0.61 () {56 vars in 1120 bytes} [Mon Aug 22 19:06:59 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:06:59.869557 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4b753f-3796-4bd3-887a-48a76a0040d2 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:06:59.968169 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4b753f-3796-4bd3-887a-48a76a0040d2 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:07:00.000683 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73781162-e155-44e7-8e40-d3609999a6ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:07:00.001288 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73781162-e155-44e7-8e40-d3609999a6ab None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:00.001564 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73781162-e155-44e7-8e40-d3609999a6ab None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:00.001837 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73781162-e155-44e7-8e40-d3609999a6ab None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:00.002500 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73781162-e155-44e7-8e40-d3609999a6ab None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:07:00.002801 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73781162-e155-44e7-8e40-d3609999a6ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:07:00.004197 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73781162-e155-44e7-8e40-d3609999a6ab None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:07:00.012051 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 554/1096] 10.222.0.61 () {56 vars in 1115 bytes} [Mon Aug 22 19:06:59 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:07:00.018127 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4556df2-2ee9-4fb7-88d5-d4595f531425 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:07:00.065086 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:07:00.120297 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4556df2-2ee9-4fb7-88d5-d4595f531425 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:07:00.158004 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fee304-a30e-4e57-a2c1-0570ee688edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:07:00.158639 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fee304-a30e-4e57-a2c1-0570ee688edf None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:00.158919 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fee304-a30e-4e57-a2c1-0570ee688edf None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:00.159190 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fee304-a30e-4e57-a2c1-0570ee688edf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:00.169819 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fee304-a30e-4e57-a2c1-0570ee688edf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:07:00.171278 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fee304-a30e-4e57-a2c1-0570ee688edf None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:07:00.185268 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-48fee304-a30e-4e57-a2c1-0570ee688edf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:07:00.185268 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:07:00.185268 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:07:00.185268 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:07:00.185268 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:07:00.185268 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:07:00.185268 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:07:00.185268 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:07:00.185268 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:07:00.185268 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:07:00.185268 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:07:00.185268 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:07:00.185268 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 19:07:00.185268 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 19:07:00.185268 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 19:07:00.185268 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 19:07:00.186386 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:07:00.186386 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:07:00.186386 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:07:00.186386 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:07:00.186386 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:07:00.186386 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:07:00.186386 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:07:00.186386 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:07:00.186386 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:07:00.186386 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:07:00.186386 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:07:00.186386 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:07:00.186386 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:07:00.186386 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:07:00.186386 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:07:00.186386 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:07:00.186386 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 19:07:00.187557 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 19:07:00.187557 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:07:00.187557 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:07:00.187557 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 543/1097] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:07:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 19:07:00.192350 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73781162-e155-44e7-8e40-d3609999a6ab None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:07:00.223597 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:07:00.293327 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73781162-e155-44e7-8e40-d3609999a6ab None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:07:00.327745 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eb2325-c572-4037-bb73-d2707b9f223b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:07:00.328357 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eb2325-c572-4037-bb73-d2707b9f223b None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:00.328640 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eb2325-c572-4037-bb73-d2707b9f223b None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:00.328914 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eb2325-c572-4037-bb73-d2707b9f223b None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:00.329558 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1eb2325-c572-4037-bb73-d2707b9f223b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:07:00.329862 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1eb2325-c572-4037-bb73-d2707b9f223b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:07:00.331226 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1eb2325-c572-4037-bb73-d2707b9f223b None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:07:00.342620 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 555/1098] 10.222.0.61 () {56 vars in 1109 bytes} [Mon Aug 22 19:07:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 19:07:00.348142 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fee304-a30e-4e57-a2c1-0570ee688edf None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:07:00.442106 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:07:00.446549 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fee304-a30e-4e57-a2c1-0570ee688edf None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:07:00.479176 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb626296-5072-466f-a107-9ffbe1d5b99b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:07:00.479765 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb626296-5072-466f-a107-9ffbe1d5b99b None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:00.480046 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb626296-5072-466f-a107-9ffbe1d5b99b None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:00.480317 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb626296-5072-466f-a107-9ffbe1d5b99b None admin] PATH_INFO: `/v3/users/nova` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:00.487540 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb626296-5072-466f-a107-9ffbe1d5b99b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:07:00.489269 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb626296-5072-466f-a107-9ffbe1d5b99b None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:07:00.500788 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-bb626296-5072-466f-a107-9ffbe1d5b99b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 22 19:07:00.500788 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:07:00.500788 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:07:00.500788 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:07:00.500788 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:07:00.500788 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:07:00.500788 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:07:00.500788 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:07:00.500788 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:07:00.500788 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:07:00.500788 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:07:00.500788 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:07:00.500788 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 19:07:00.500788 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 19:07:00.500788 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 19:07:00.500788 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 19:07:00.501733 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:07:00.501733 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:07:00.501733 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 19:07:00.501733 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 19:07:00.501733 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 19:07:00.501733 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 19:07:00.501733 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:07:00.501733 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:07:00.501733 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:07:00.501733 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:07:00.501733 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:07:00.501733 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:07:00.501733 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:07:00.501733 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:07:00.501733 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:07:00.501733 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:07:00.501733 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:07:00.501733 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:07:00.502727 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:07:00.502727 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:07:00.502727 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 19:07:00.502727 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 19:07:00.502727 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 19:07:00.502727 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 19:07:00.502727 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 19:07:00.502727 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 19:07:00.502727 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 22 19:07:00.502727 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:07:00.502727 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 544/1099] 10.222.0.61 () {56 vars in 1093 bytes} [Mon Aug 22 19:07:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:07:00.507801 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eb2325-c572-4037-bb73-d2707b9f223b None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:07:00.596246 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:07:00.606097 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eb2325-c572-4037-bb73-d2707b9f223b None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:07:00.640968 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a06af6-f22b-44eb-9295-6d902f69e832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:07:00.641576 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a06af6-f22b-44eb-9295-6d902f69e832 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:00.641860 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a06af6-f22b-44eb-9295-6d902f69e832 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:00.642136 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a06af6-f22b-44eb-9295-6d902f69e832 None admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:00.642866 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a06af6-f22b-44eb-9295-6d902f69e832 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:07:00.643174 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a06af6-f22b-44eb-9295-6d902f69e832 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:07:00.644630 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a06af6-f22b-44eb-9295-6d902f69e832 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:07:00.661507 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 556/1100] 10.222.0.61 () {56 vars in 1133 bytes} [Mon Aug 22 19:07:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:07:00.667274 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb626296-5072-466f-a107-9ffbe1d5b99b None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:07:00.767099 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb626296-5072-466f-a107-9ffbe1d5b99b None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:07:00.799495 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96691dad-1288-45f5-8ec2-b43e33ba7747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:07:00.800127 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96691dad-1288-45f5-8ec2-b43e33ba7747 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:00.800405 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96691dad-1288-45f5-8ec2-b43e33ba7747 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:00.800673 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96691dad-1288-45f5-8ec2-b43e33ba7747 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:00.826773 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:07:00.867176 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96691dad-1288-45f5-8ec2-b43e33ba7747 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:07:00.868633 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96691dad-1288-45f5-8ec2-b43e33ba7747 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:07:00.963915 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 545/1101] 10.222.0.61 () {58 vars in 1282 bytes} [Mon Aug 22 19:07:00 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 22 19:07:00.970187 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a06af6-f22b-44eb-9295-6d902f69e832 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:07:01.066371 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a06af6-f22b-44eb-9295-6d902f69e832 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:07:01.101318 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358fa281-f6c1-4648-9b8c-036e44214094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:07:01.101948 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358fa281-f6c1-4648-9b8c-036e44214094 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:01.102222 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358fa281-f6c1-4648-9b8c-036e44214094 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:01.102493 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358fa281-f6c1-4648-9b8c-036e44214094 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:01.115713 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358fa281-f6c1-4648-9b8c-036e44214094 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:07:01.117254 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358fa281-f6c1-4648-9b8c-036e44214094 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:07:01.125841 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:07:01.132456 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-358fa281-f6c1-4648-9b8c-036e44214094 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:07:01.132456 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:07:01.132456 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:07:01.132456 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:07:01.132456 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:07:01.132456 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:07:01.132456 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:07:01.132456 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:07:01.132456 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:07:01.132456 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:07:01.132456 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:07:01.132456 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:07:01.132456 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 19:07:01.132456 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 19:07:01.132456 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 19:07:01.132456 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 19:07:01.133560 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:07:01.133560 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:07:01.133560 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:07:01.133560 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:07:01.133560 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:07:01.133560 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:07:01.133560 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:07:01.133560 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:07:01.133560 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:07:01.133560 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:07:01.133560 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:07:01.133560 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:07:01.133560 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:07:01.133560 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:07:01.133560 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:07:01.133560 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:07:01.133560 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 19:07:01.134559 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 19:07:01.134559 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:07:01.134559 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:07:01.135967 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 557/1102] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:07:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:07:01.139344 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96691dad-1288-45f5-8ec2-b43e33ba7747 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:07:01.197172 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:07:01.239510 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96691dad-1288-45f5-8ec2-b43e33ba7747 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:07:01.273440 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b4da4b-fc03-44a0-9723-6bc4d2db0f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:07:01.274042 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b4da4b-fc03-44a0-9723-6bc4d2db0f59 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:01.274320 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b4da4b-fc03-44a0-9723-6bc4d2db0f59 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:01.274593 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b4da4b-fc03-44a0-9723-6bc4d2db0f59 None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:01.275231 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b4da4b-fc03-44a0-9723-6bc4d2db0f59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:07:01.275531 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b4da4b-fc03-44a0-9723-6bc4d2db0f59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:07:01.276933 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b4da4b-fc03-44a0-9723-6bc4d2db0f59 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:07:01.288322 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 546/1103] 10.222.0.61 () {56 vars in 1109 bytes} [Mon Aug 22 19:07:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:07:01.293944 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358fa281-f6c1-4648-9b8c-036e44214094 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:07:01.396909 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358fa281-f6c1-4648-9b8c-036e44214094 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:07:01.430525 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35cc5f4-afb2-4ae0-9ced-4f95c2595db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:07:01.431161 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35cc5f4-afb2-4ae0-9ced-4f95c2595db1 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:01.431442 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35cc5f4-afb2-4ae0-9ced-4f95c2595db1 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:01.431746 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35cc5f4-afb2-4ae0-9ced-4f95c2595db1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:01.438857 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35cc5f4-afb2-4ae0-9ced-4f95c2595db1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:07:01.440332 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35cc5f4-afb2-4ae0-9ced-4f95c2595db1 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:07:01.451378 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-d35cc5f4-afb2-4ae0-9ced-4f95c2595db1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 19:07:01.451378 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:07:01.451378 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:07:01.451378 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:07:01.451378 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:07:01.451378 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:07:01.451378 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:07:01.451378 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:07:01.451378 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:07:01.451378 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:07:01.451378 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:07:01.451378 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:07:01.451378 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 19:07:01.451378 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 19:07:01.451378 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 19:07:01.452462 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 19:07:01.452462 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:07:01.452462 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:07:01.452462 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:07:01.452462 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:07:01.452462 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:07:01.452462 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:07:01.452462 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:07:01.452462 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:07:01.452462 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:07:01.452462 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:07:01.452462 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:07:01.452462 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:07:01.452462 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:07:01.452462 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:07:01.452462 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:07:01.452462 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:07:01.453556 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 19:07:01.453556 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 19:07:01.453556 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 19:07:01.453556 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 19:07:01.453556 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 19:07:01.453556 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 19:07:01.453556 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 19:07:01.453556 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:07:01.455801 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 558/1104] 10.222.0.61 () {56 vars in 1111 bytes} [Mon Aug 22 19:07:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:07:01.458481 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b4da4b-fc03-44a0-9723-6bc4d2db0f59 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:07:01.558300 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d201896-572b-4268-81f7-652a123f9439 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:01.558832 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d201896-572b-4268-81f7-652a123f9439 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:01.558832 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d201896-572b-4268-81f7-652a123f9439 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:01.559244 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b4da4b-fc03-44a0-9723-6bc4d2db0f59 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:07:01.560194 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 559/1105] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:07:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 19:07:01.565607 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45489403-d278-42ab-a038-3b5e71db956b None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:01.565882 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45489403-d278-42ab-a038-3b5e71db956b None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:01.566151 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45489403-d278-42ab-a038-3b5e71db956b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:01.577464 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:07:01.583860 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-45489403-d278-42ab-a038-3b5e71db956b None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:07:01.585045 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-45489403-d278-42ab-a038-3b5e71db956b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:07:01.594629 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e584d1-0fef-478b-8414-2eb72e64b2ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:07:01.595228 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e584d1-0fef-478b-8414-2eb72e64b2ff None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:01.595502 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e584d1-0fef-478b-8414-2eb72e64b2ff None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:01.595811 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e584d1-0fef-478b-8414-2eb72e64b2ff None admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:01.596500 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e584d1-0fef-478b-8414-2eb72e64b2ff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:07:01.596800 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e584d1-0fef-478b-8414-2eb72e64b2ff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:07:01.598210 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e584d1-0fef-478b-8414-2eb72e64b2ff None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:07:01.611039 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 547/1106] 10.222.0.61 () {56 vars in 1148 bytes} [Mon Aug 22 19:07:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:07:01.616990 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e584d1-0fef-478b-8414-2eb72e64b2ff None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:07:01.641606 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:07:01.653884 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 560/1107] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:07:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 22 19:07:01.717272 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e584d1-0fef-478b-8414-2eb72e64b2ff None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:07:01.749959 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3290e7a2-ffc4-4e59-a83e-123dff40e6f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:07:01.750547 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3290e7a2-ffc4-4e59-a83e-123dff40e6f3 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:01.750820 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3290e7a2-ffc4-4e59-a83e-123dff40e6f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:01.751087 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3290e7a2-ffc4-4e59-a83e-123dff40e6f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:01.751901 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3290e7a2-ffc4-4e59-a83e-123dff40e6f3 None admin] RBAC: Adding query filter params (role.id=aa9eb04acf884c51ad2adb05380bc7a1, scope.project.id=e07dc5ebdc504c6da706a2d21c0a3970, user.id=e7f43e2756b746a6825ca1a06b7d7c91) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:07:01.752203 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3290e7a2-ffc4-4e59-a83e-123dff40e6f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:07:01.753611 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3290e7a2-ffc4-4e59-a83e-123dff40e6f3 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:07:01.759162 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 548/1108] 10.222.0.61 () {56 vars in 1374 bytes} [Mon Aug 22 19:07:01 2022] GET /identity/v3/role_assignments?user.id=e7f43e2756b746a6825ca1a06b7d7c91&scope.project.id=e07dc5ebdc504c6da706a2d21c0a3970&role.id=aa9eb04acf884c51ad2adb05380bc7a1 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 19:07:01.921711 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45489403-d278-42ab-a038-3b5e71db956b None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:07:02.023854 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45489403-d278-42ab-a038-3b5e71db956b None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:07:02.058285 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60029cce-223d-4944-baee-4d338050b8cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:07:02.058923 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60029cce-223d-4944-baee-4d338050b8cf None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:02.059206 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60029cce-223d-4944-baee-4d338050b8cf None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:02.059477 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60029cce-223d-4944-baee-4d338050b8cf None admin] PATH_INFO: `/v3/domains/default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:02.060653 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60029cce-223d-4944-baee-4d338050b8cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:07:02.062082 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60029cce-223d-4944-baee-4d338050b8cf None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:07:02.063668 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 561/1109] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:07:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:07:02.069535 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3290e7a2-ffc4-4e59-a83e-123dff40e6f3 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:07:02.120758 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:07:02.180741 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3290e7a2-ffc4-4e59-a83e-123dff40e6f3 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:07:02.216139 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bb017c-52c0-40ff-a528-a940cd20404e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:07:02.216748 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bb017c-52c0-40ff-a528-a940cd20404e None admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:02.217027 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bb017c-52c0-40ff-a528-a940cd20404e None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:02.217290 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bb017c-52c0-40ff-a528-a940cd20404e None admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:02.217941 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bb017c-52c0-40ff-a528-a940cd20404e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:07:02.219351 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bb017c-52c0-40ff-a528-a940cd20404e None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:07:02.242605 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 549/1110] 10.222.0.61 () {60 vars in 1132 bytes} [Mon Aug 22 19:07:02 2022] POST /identity/v3/users => generated 286 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 19:07:04.194164 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0317b1cb-f5aa-4536-beb8-3ac5ded3e1ee None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:04.194610 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0317b1cb-f5aa-4536-beb8-3ac5ded3e1ee None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:04.194700 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0317b1cb-f5aa-4536-beb8-3ac5ded3e1ee None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:04.196088 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 562/1111] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:07:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:07:04.201145 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a064de5-e88b-4fe7-8c52-4208bc38d132 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:04.201560 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a064de5-e88b-4fe7-8c52-4208bc38d132 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:04.201711 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a064de5-e88b-4fe7-8c52-4208bc38d132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:04.219906 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-6a064de5-e88b-4fe7-8c52-4208bc38d132 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:07:04.221100 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-6a064de5-e88b-4fe7-8c52-4208bc38d132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:07:04.285352 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 550/1112] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:07:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 19:07:04.554038 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0317b1cb-f5aa-4536-beb8-3ac5ded3e1ee None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:07:04.612270 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:07:04.654769 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0317b1cb-f5aa-4536-beb8-3ac5ded3e1ee None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:07:04.686808 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb4a0dc-92a2-453f-b6f1-a268875c89e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:07:04.687424 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb4a0dc-92a2-453f-b6f1-a268875c89e9 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:04.687727 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb4a0dc-92a2-453f-b6f1-a268875c89e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:04.688003 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb4a0dc-92a2-453f-b6f1-a268875c89e9 None admin] PATH_INFO: `/v3/roles/9f0952608a704dfca5fdbe4046331f21` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:04.689390 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb4a0dc-92a2-453f-b6f1-a268875c89e9 None admin] RBAC: Authorizing `identity:get_role(role_id=9f0952608a704dfca5fdbe4046331f21)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:07:04.690742 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb4a0dc-92a2-453f-b6f1-a268875c89e9 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:07:04.692076 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 563/1113] 10.222.0.61 () {56 vars in 1177 bytes} [Mon Aug 22 19:07:04 2022] GET /identity/v3/roles/9f0952608a704dfca5fdbe4046331f21 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:07:04.697828 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a064de5-e88b-4fe7-8c52-4208bc38d132 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:07:04.725795 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:07:04.795715 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a064de5-e88b-4fe7-8c52-4208bc38d132 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:07:04.828823 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15ce3b2-07b8-4530-a11e-dd4f44b691c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:07:04.829382 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15ce3b2-07b8-4530-a11e-dd4f44b691c1 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:04.829661 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15ce3b2-07b8-4530-a11e-dd4f44b691c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:04.829935 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15ce3b2-07b8-4530-a11e-dd4f44b691c1 None admin] PATH_INFO: `/v3/users/584576d0d4cd446b9726a51a48a51d5a` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:04.843182 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15ce3b2-07b8-4530-a11e-dd4f44b691c1 None admin] RBAC: Authorizing `identity:get_user(user_id=584576d0d4cd446b9726a51a48a51d5a)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:07:04.844629 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15ce3b2-07b8-4530-a11e-dd4f44b691c1 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:07:04.846193 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 551/1114] 10.222.0.61 () {56 vars in 1177 bytes} [Mon Aug 22 19:07:04 2022] GET /identity/v3/users/584576d0d4cd446b9726a51a48a51d5a => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:07:04.851768 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb4a0dc-92a2-453f-b6f1-a268875c89e9 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:07:04.948378 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb4a0dc-92a2-453f-b6f1-a268875c89e9 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:07:04.973169 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:07:04.981844 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253890ad-ce66-493c-bc70-642df182d45a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:07:04.982438 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253890ad-ce66-493c-bc70-642df182d45a None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:04.982717 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253890ad-ce66-493c-bc70-642df182d45a None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:04.982990 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253890ad-ce66-493c-bc70-642df182d45a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:05.047394 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253890ad-ce66-493c-bc70-642df182d45a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:07:05.048845 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253890ad-ce66-493c-bc70-642df182d45a None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:07:05.146516 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 564/1115] 10.222.0.61 () {58 vars in 1282 bytes} [Mon Aug 22 19:07:04 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 22 19:07:05.152441 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15ce3b2-07b8-4530-a11e-dd4f44b691c1 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:07:05.234129 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:07:05.248903 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15ce3b2-07b8-4530-a11e-dd4f44b691c1 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:07:05.281984 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b43d977-5c62-40aa-8afd-3de2e9a623f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:07:05.282597 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b43d977-5c62-40aa-8afd-3de2e9a623f1 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:05.282875 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b43d977-5c62-40aa-8afd-3de2e9a623f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:05.283147 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b43d977-5c62-40aa-8afd-3de2e9a623f1 None admin] PATH_INFO: `/v3/projects/9d2c1e4458e84033a49b2bee6c1af26e` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:05.284264 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b43d977-5c62-40aa-8afd-3de2e9a623f1 None admin] RBAC: Authorizing `identity:get_project(project_id=9d2c1e4458e84033a49b2bee6c1af26e)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:07:05.285678 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b43d977-5c62-40aa-8afd-3de2e9a623f1 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:07:05.287279 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 552/1116] 10.222.0.61 () {56 vars in 1186 bytes} [Mon Aug 22 19:07:05 2022] GET /identity/v3/projects/9d2c1e4458e84033a49b2bee6c1af26e => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:07:05.293295 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253890ad-ce66-493c-bc70-642df182d45a None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:07:05.318769 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:07:05.389117 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253890ad-ce66-493c-bc70-642df182d45a None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:07:05.421468 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2025161-1895-48ef-9b0c-959227e8f932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:07:05.421995 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2025161-1895-48ef-9b0c-959227e8f932 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:05.422273 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2025161-1895-48ef-9b0c-959227e8f932 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:05.422538 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2025161-1895-48ef-9b0c-959227e8f932 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:05.423249 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2025161-1895-48ef-9b0c-959227e8f932 None admin] RBAC: Adding query filter params (role.id=9f0952608a704dfca5fdbe4046331f21, scope.project.id=9d2c1e4458e84033a49b2bee6c1af26e, user.id=584576d0d4cd446b9726a51a48a51d5a) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:07:05.423550 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2025161-1895-48ef-9b0c-959227e8f932 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:07:05.424979 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2025161-1895-48ef-9b0c-959227e8f932 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:07:05.430107 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 565/1117] 10.222.0.61 () {56 vars in 1374 bytes} [Mon Aug 22 19:07:05 2022] GET /identity/v3/role_assignments?user.id=584576d0d4cd446b9726a51a48a51d5a&scope.project.id=9d2c1e4458e84033a49b2bee6c1af26e&role.id=9f0952608a704dfca5fdbe4046331f21 => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:07:07.283660 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ecced4-c856-40af-b48c-38a275947038 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:07.284185 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ecced4-c856-40af-b48c-38a275947038 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:07.284302 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ecced4-c856-40af-b48c-38a275947038 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:07.285565 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 553/1118] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:07:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:07:07.290730 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f4a118-5144-48c0-883e-4cfebb8fe1c1 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:07.291142 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f4a118-5144-48c0-883e-4cfebb8fe1c1 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:07.291269 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f4a118-5144-48c0-883e-4cfebb8fe1c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:07.308745 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-d4f4a118-5144-48c0-883e-4cfebb8fe1c1 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:07:07.309915 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-d4f4a118-5144-48c0-883e-4cfebb8fe1c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:07:07.374967 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 566/1119] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:07:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 19:07:07.648757 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ecced4-c856-40af-b48c-38a275947038 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:07:07.747581 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ecced4-c856-40af-b48c-38a275947038 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:07:07.779911 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a967f0-a4ac-4215-8d5b-d85253d4f4b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:07:07.780525 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a967f0-a4ac-4215-8d5b-d85253d4f4b6 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:07.780798 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a967f0-a4ac-4215-8d5b-d85253d4f4b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:07.781066 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a967f0-a4ac-4215-8d5b-d85253d4f4b6 None admin] PATH_INFO: `/v3/roles/9f0952608a704dfca5fdbe4046331f21` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:07.782439 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a967f0-a4ac-4215-8d5b-d85253d4f4b6 None admin] RBAC: Authorizing `identity:get_role(role_id=9f0952608a704dfca5fdbe4046331f21)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:07:07.783805 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a967f0-a4ac-4215-8d5b-d85253d4f4b6 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:07:07.785183 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 554/1120] 10.222.0.61 () {56 vars in 1177 bytes} [Mon Aug 22 19:07:07 2022] GET /identity/v3/roles/9f0952608a704dfca5fdbe4046331f21 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 19:07:07.790822 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f4a118-5144-48c0-883e-4cfebb8fe1c1 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:07:07.811790 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:07:07.886055 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f4a118-5144-48c0-883e-4cfebb8fe1c1 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:07:07.918437 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d46f9a7-3be8-409a-879d-2528f6729ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:07:07.919000 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d46f9a7-3be8-409a-879d-2528f6729ad5 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:07.919275 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d46f9a7-3be8-409a-879d-2528f6729ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:07.919545 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d46f9a7-3be8-409a-879d-2528f6729ad5 None admin] PATH_INFO: `/v3/users/584576d0d4cd446b9726a51a48a51d5a` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:07.921253 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d46f9a7-3be8-409a-879d-2528f6729ad5 None admin] RBAC: Authorizing `identity:get_user(user_id=584576d0d4cd446b9726a51a48a51d5a)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:07:07.922662 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d46f9a7-3be8-409a-879d-2528f6729ad5 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:07:07.924225 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 567/1121] 10.222.0.61 () {56 vars in 1177 bytes} [Mon Aug 22 19:07:07 2022] GET /identity/v3/users/584576d0d4cd446b9726a51a48a51d5a => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:07:07.929760 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a967f0-a4ac-4215-8d5b-d85253d4f4b6 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:07:07.953033 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:07:08.026657 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a967f0-a4ac-4215-8d5b-d85253d4f4b6 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:07:08.059953 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935f2098-37c3-474f-8e79-3b0a36ea190d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:07:08.060593 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935f2098-37c3-474f-8e79-3b0a36ea190d None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:08.060873 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935f2098-37c3-474f-8e79-3b0a36ea190d None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:08.061147 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935f2098-37c3-474f-8e79-3b0a36ea190d None admin] PATH_INFO: `/v3/projects/9d2c1e4458e84033a49b2bee6c1af26e` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:08.062261 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935f2098-37c3-474f-8e79-3b0a36ea190d None admin] RBAC: Authorizing `identity:get_project(project_id=9d2c1e4458e84033a49b2bee6c1af26e)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:07:08.063687 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935f2098-37c3-474f-8e79-3b0a36ea190d None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:07:08.065299 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 555/1122] 10.222.0.61 () {56 vars in 1186 bytes} [Mon Aug 22 19:07:07 2022] GET /identity/v3/projects/9d2c1e4458e84033a49b2bee6c1af26e => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:07:08.070922 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d46f9a7-3be8-409a-879d-2528f6729ad5 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:07:08.156152 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:07:08.167958 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d46f9a7-3be8-409a-879d-2528f6729ad5 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:07:08.202472 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f26366-38ca-4da2-82a1-c56b32a7619e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:07:08.203115 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f26366-38ca-4da2-82a1-c56b32a7619e None admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:08.203390 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f26366-38ca-4da2-82a1-c56b32a7619e None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:08.203681 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f26366-38ca-4da2-82a1-c56b32a7619e None admin] PATH_INFO: `/v3/projects/9d2c1e4458e84033a49b2bee6c1af26e/users/584576d0d4cd446b9726a51a48a51d5a/roles/9f0952608a704dfca5fdbe4046331f21` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:08.205791 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f26366-38ca-4da2-82a1-c56b32a7619e None admin] RBAC: Authorizing `identity:create_grant(project_id=9d2c1e4458e84033a49b2bee6c1af26e, user_id=584576d0d4cd446b9726a51a48a51d5a, role_id=9f0952608a704dfca5fdbe4046331f21)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:07:08.207208 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f26366-38ca-4da2-82a1-c56b32a7619e None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:07:08.219711 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 568/1123] 10.222.0.61 () {58 vars in 1439 bytes} [Mon Aug 22 19:07:08 2022] PUT /identity/v3/projects/9d2c1e4458e84033a49b2bee6c1af26e/users/584576d0d4cd446b9726a51a48a51d5a/roles/9f0952608a704dfca5fdbe4046331f21 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:07:10.058764 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ec9068-f480-4a0e-93db-90f1412f614e None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:10.059372 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ec9068-f480-4a0e-93db-90f1412f614e None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:10.059372 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ec9068-f480-4a0e-93db-90f1412f614e None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:10.061199 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 556/1124] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:07:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:07:10.066354 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4411a01-ece5-4bcf-9f54-d65d7072668c None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:10.066778 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4411a01-ece5-4bcf-9f54-d65d7072668c None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:10.066900 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4411a01-ece5-4bcf-9f54-d65d7072668c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:10.084978 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-d4411a01-ece5-4bcf-9f54-d65d7072668c None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:07:10.086157 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-d4411a01-ece5-4bcf-9f54-d65d7072668c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:07:10.151585 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 569/1125] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:07:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 19:07:10.416503 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ec9068-f480-4a0e-93db-90f1412f614e None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:07:10.505788 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:07:10.516620 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ec9068-f480-4a0e-93db-90f1412f614e None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:07:10.549481 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75022f4a-a26d-465f-b92c-dc5d4565237c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:07:10.550107 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75022f4a-a26d-465f-b92c-dc5d4565237c None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:10.550386 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75022f4a-a26d-465f-b92c-dc5d4565237c None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:10.550661 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75022f4a-a26d-465f-b92c-dc5d4565237c None admin] PATH_INFO: `/v3/roles/9f0952608a704dfca5fdbe4046331f21` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:10.552090 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75022f4a-a26d-465f-b92c-dc5d4565237c None admin] RBAC: Authorizing `identity:get_role(role_id=9f0952608a704dfca5fdbe4046331f21)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:07:10.553444 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75022f4a-a26d-465f-b92c-dc5d4565237c None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:07:10.554743 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 557/1126] 10.222.0.61 () {56 vars in 1177 bytes} [Mon Aug 22 19:07:10 2022] GET /identity/v3/roles/9f0952608a704dfca5fdbe4046331f21 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:07:10.561031 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4411a01-ece5-4bcf-9f54-d65d7072668c None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:07:10.636598 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:07:10.658170 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4411a01-ece5-4bcf-9f54-d65d7072668c None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:07:10.694183 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b8a201-ea1d-4b4e-ae55-38eb5dea324b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:07:10.694806 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b8a201-ea1d-4b4e-ae55-38eb5dea324b None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:10.695083 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b8a201-ea1d-4b4e-ae55-38eb5dea324b None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:10.695351 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b8a201-ea1d-4b4e-ae55-38eb5dea324b None admin] PATH_INFO: `/v3/users/584576d0d4cd446b9726a51a48a51d5a` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:10.696495 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b8a201-ea1d-4b4e-ae55-38eb5dea324b None admin] RBAC: Authorizing `identity:get_user(user_id=584576d0d4cd446b9726a51a48a51d5a)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:07:10.697911 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b8a201-ea1d-4b4e-ae55-38eb5dea324b None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:07:10.699473 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 570/1127] 10.222.0.61 () {56 vars in 1177 bytes} [Mon Aug 22 19:07:10 2022] GET /identity/v3/users/584576d0d4cd446b9726a51a48a51d5a => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:07:10.705033 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75022f4a-a26d-465f-b92c-dc5d4565237c None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:07:10.801224 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75022f4a-a26d-465f-b92c-dc5d4565237c None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:07:10.833276 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22362532-3739-4495-ab1d-4020bff05f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:07:10.833862 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22362532-3739-4495-ab1d-4020bff05f32 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:10.834138 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22362532-3739-4495-ab1d-4020bff05f32 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:10.834408 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22362532-3739-4495-ab1d-4020bff05f32 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:10.860693 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:07:10.899931 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22362532-3739-4495-ab1d-4020bff05f32 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:07:10.901345 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22362532-3739-4495-ab1d-4020bff05f32 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:07:10.996185 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 558/1128] 10.222.0.61 () {58 vars in 1282 bytes} [Mon Aug 22 19:07:10 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 22 19:07:11.001967 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b8a201-ea1d-4b4e-ae55-38eb5dea324b None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:07:11.098319 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b8a201-ea1d-4b4e-ae55-38eb5dea324b None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:07:11.131006 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a783fcb4-9810-4a75-8dff-ca077d52bcfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:07:11.131634 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a783fcb4-9810-4a75-8dff-ca077d52bcfa None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:11.131937 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a783fcb4-9810-4a75-8dff-ca077d52bcfa None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:11.132209 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a783fcb4-9810-4a75-8dff-ca077d52bcfa None admin] PATH_INFO: `/v3/projects/9d2c1e4458e84033a49b2bee6c1af26e` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:11.133321 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a783fcb4-9810-4a75-8dff-ca077d52bcfa None admin] RBAC: Authorizing `identity:get_project(project_id=9d2c1e4458e84033a49b2bee6c1af26e)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:07:11.134743 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a783fcb4-9810-4a75-8dff-ca077d52bcfa None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:07:11.136391 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 571/1129] 10.222.0.61 () {56 vars in 1186 bytes} [Mon Aug 22 19:07:11 2022] GET /identity/v3/projects/9d2c1e4458e84033a49b2bee6c1af26e => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:07:11.142940 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22362532-3739-4495-ab1d-4020bff05f32 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:07:11.198741 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:07:11.239567 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22362532-3739-4495-ab1d-4020bff05f32 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:07:11.272379 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7627fe30-c01f-482c-92a6-ed7e64144118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:07:11.272908 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7627fe30-c01f-482c-92a6-ed7e64144118 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:11.273183 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7627fe30-c01f-482c-92a6-ed7e64144118 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:11.273451 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7627fe30-c01f-482c-92a6-ed7e64144118 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:11.274198 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7627fe30-c01f-482c-92a6-ed7e64144118 None admin] RBAC: Adding query filter params (role.id=9f0952608a704dfca5fdbe4046331f21, scope.project.id=9d2c1e4458e84033a49b2bee6c1af26e, user.id=584576d0d4cd446b9726a51a48a51d5a) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:07:11.274501 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7627fe30-c01f-482c-92a6-ed7e64144118 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:07:11.275932 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7627fe30-c01f-482c-92a6-ed7e64144118 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:07:11.282071 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 559/1130] 10.222.0.61 () {56 vars in 1374 bytes} [Mon Aug 22 19:07:11 2022] GET /identity/v3/role_assignments?user.id=584576d0d4cd446b9726a51a48a51d5a&scope.project.id=9d2c1e4458e84033a49b2bee6c1af26e&role.id=9f0952608a704dfca5fdbe4046331f21 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:07:13.166159 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9590e3-780b-4e6a-bec8-2feac59a099a None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:13.166567 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9590e3-780b-4e6a-bec8-2feac59a099a None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:13.166706 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9590e3-780b-4e6a-bec8-2feac59a099a None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:13.168095 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 572/1131] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:07:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:07:13.173309 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03d8a1c-d807-4f53-b2b3-6ed48a45088c None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:13.173740 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03d8a1c-d807-4f53-b2b3-6ed48a45088c None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:13.173859 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03d8a1c-d807-4f53-b2b3-6ed48a45088c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:13.191961 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-a03d8a1c-d807-4f53-b2b3-6ed48a45088c None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:07:13.193149 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-a03d8a1c-d807-4f53-b2b3-6ed48a45088c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:07:13.257298 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 560/1132] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:07:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 19:07:13.525272 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9590e3-780b-4e6a-bec8-2feac59a099a None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:07:13.541557 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:07:13.627241 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9590e3-780b-4e6a-bec8-2feac59a099a None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:07:13.660183 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f53314-470a-4d82-9424-79eddb1d2512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:07:13.660813 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f53314-470a-4d82-9424-79eddb1d2512 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:13.661090 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f53314-470a-4d82-9424-79eddb1d2512 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:13.661465 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f53314-470a-4d82-9424-79eddb1d2512 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:13.662469 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f53314-470a-4d82-9424-79eddb1d2512 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:07:13.663923 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f53314-470a-4d82-9424-79eddb1d2512 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:07:13.665476 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 573/1133] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:07:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:07:13.671740 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03d8a1c-d807-4f53-b2b3-6ed48a45088c None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:07:13.721595 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:07:13.766959 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03d8a1c-d807-4f53-b2b3-6ed48a45088c None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:07:13.799773 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43cb484-994d-409d-818b-5a469f57a3ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:07:13.800308 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43cb484-994d-409d-818b-5a469f57a3ae None admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:13.800582 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43cb484-994d-409d-818b-5a469f57a3ae None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:13.800849 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43cb484-994d-409d-818b-5a469f57a3ae None admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:13.801487 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43cb484-994d-409d-818b-5a469f57a3ae None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:07:13.802886 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43cb484-994d-409d-818b-5a469f57a3ae None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:07:13.828142 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 561/1134] 10.222.0.61 () {60 vars in 1141 bytes} [Mon Aug 22 19:07:13 2022] POST /identity/v3/projects => generated 311 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:07:15.709939 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44da7f4c-e532-47e3-8bde-6315d0801d6b None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:15.710504 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44da7f4c-e532-47e3-8bde-6315d0801d6b None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:15.710504 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44da7f4c-e532-47e3-8bde-6315d0801d6b None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:15.711863 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 574/1135] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:07:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:07:15.717565 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985682e8-4779-474e-9c73-e6d260276843 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:15.718002 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985682e8-4779-474e-9c73-e6d260276843 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:15.718095 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985682e8-4779-474e-9c73-e6d260276843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:15.735805 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-985682e8-4779-474e-9c73-e6d260276843 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:07:15.736983 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-985682e8-4779-474e-9c73-e6d260276843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:07:15.801107 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 562/1136] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:07:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 19:07:16.065547 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44da7f4c-e532-47e3-8bde-6315d0801d6b None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:07:16.149477 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:07:16.168471 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44da7f4c-e532-47e3-8bde-6315d0801d6b None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:07:16.202179 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22842b18-8344-40a9-b61d-9c35e3e06e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:07:16.202811 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22842b18-8344-40a9-b61d-9c35e3e06e7b None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:16.203091 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22842b18-8344-40a9-b61d-9c35e3e06e7b None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:16.203365 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22842b18-8344-40a9-b61d-9c35e3e06e7b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:16.204517 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22842b18-8344-40a9-b61d-9c35e3e06e7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:07:16.205939 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22842b18-8344-40a9-b61d-9c35e3e06e7b None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:07:16.207501 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 575/1137] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:07:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:07:16.213159 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985682e8-4779-474e-9c73-e6d260276843 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:07:16.245562 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:07:16.310965 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985682e8-4779-474e-9c73-e6d260276843 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:07:16.344520 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffff64a5-84cc-486d-834c-9a9ef07578b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:07:16.345054 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffff64a5-84cc-486d-834c-9a9ef07578b7 None admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:16.345333 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffff64a5-84cc-486d-834c-9a9ef07578b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:16.345605 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffff64a5-84cc-486d-834c-9a9ef07578b7 None admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:16.346254 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffff64a5-84cc-486d-834c-9a9ef07578b7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:07:16.347691 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffff64a5-84cc-486d-834c-9a9ef07578b7 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:07:16.368251 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 563/1138] 10.222.0.61 () {60 vars in 1132 bytes} [Mon Aug 22 19:07:16 2022] POST /identity/v3/users => generated 286 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:07:18.256310 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ffb398-e3ef-4c80-8e52-55bb1583f67a None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:18.256830 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ffb398-e3ef-4c80-8e52-55bb1583f67a None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:18.256934 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ffb398-e3ef-4c80-8e52-55bb1583f67a None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:18.258195 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 576/1139] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:07:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:07:18.263354 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c08a66a-ca4c-4f89-9c3b-0b239561f0ca None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:18.263825 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c08a66a-ca4c-4f89-9c3b-0b239561f0ca None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:18.263935 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c08a66a-ca4c-4f89-9c3b-0b239561f0ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:18.282813 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-8c08a66a-ca4c-4f89-9c3b-0b239561f0ca None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:07:18.284048 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-8c08a66a-ca4c-4f89-9c3b-0b239561f0ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:07:18.349095 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 564/1140] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:07:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 19:07:18.617373 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ffb398-e3ef-4c80-8e52-55bb1583f67a None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:07:18.717171 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ffb398-e3ef-4c80-8e52-55bb1583f67a None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:07:18.749749 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071bbf9d-514e-4946-891e-ecaa2bac2cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:07:18.750368 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071bbf9d-514e-4946-891e-ecaa2bac2cba None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:18.750647 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071bbf9d-514e-4946-891e-ecaa2bac2cba None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:18.750928 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071bbf9d-514e-4946-891e-ecaa2bac2cba None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:18.760742 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071bbf9d-514e-4946-891e-ecaa2bac2cba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 19:07:18.765989 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-071bbf9d-514e-4946-891e-ecaa2bac2cba None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 19:07:18.765989 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:07:18.765989 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:07:18.765989 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:07:18.765989 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:07:18.765989 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:07:18.765989 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:07:18.765989 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:07:18.765989 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:07:18.765989 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:07:18.765989 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:07:18.765989 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:07:18.765989 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 19:07:18.765989 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 19:07:18.765989 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 19:07:18.765989 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 19:07:18.767149 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 19:07:18.767149 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 19:07:18.767149 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 19:07:18.767149 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 19:07:18.767149 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:07:18.767149 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:07:18.767149 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:07:18.767149 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:07:18.767149 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:07:18.767149 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:07:18.767149 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:07:18.767149 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:07:18.767149 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:07:18.767149 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:07:18.767149 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:07:18.767149 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:07:18.768410 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:07:18.768410 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:07:18.768410 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:07:18.768410 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:07:18.768410 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 19:07:18.768410 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 19:07:18.768410 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 19:07:18.768410 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 19:07:18.768410 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 19:07:18.768410 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 19:07:18.768410 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 19:07:18.768410 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:07:18.768410 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 577/1141] 10.222.0.61 () {56 vars in 1096 bytes} [Mon Aug 22 19:07:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:07:18.772889 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c08a66a-ca4c-4f89-9c3b-0b239561f0ca None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:07:18.794066 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:07:18.870665 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c08a66a-ca4c-4f89-9c3b-0b239561f0ca None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:07:18.903419 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011396f4-b021-4794-9d33-baac33255b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:07:18.904695 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011396f4-b021-4794-9d33-baac33255b1d None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:18.904971 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011396f4-b021-4794-9d33-baac33255b1d None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:18.905242 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011396f4-b021-4794-9d33-baac33255b1d None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:18.905903 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011396f4-b021-4794-9d33-baac33255b1d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:07:18.906202 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011396f4-b021-4794-9d33-baac33255b1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:07:18.907551 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011396f4-b021-4794-9d33-baac33255b1d None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:07:18.915355 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 565/1142] 10.222.0.61 () {56 vars in 1099 bytes} [Mon Aug 22 19:07:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:07:18.920911 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071bbf9d-514e-4946-891e-ecaa2bac2cba None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:07:18.933366 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:07:19.019603 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071bbf9d-514e-4946-891e-ecaa2bac2cba None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:07:19.052161 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f00dda-059a-46ae-a782-765bb24db847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:07:19.052773 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f00dda-059a-46ae-a782-765bb24db847 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:19.053050 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f00dda-059a-46ae-a782-765bb24db847 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:19.053323 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f00dda-059a-46ae-a782-765bb24db847 None admin] PATH_INFO: `/v3/users/b1acb0a671c14b0fbc92babd2a80ad94` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:19.067036 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f00dda-059a-46ae-a782-765bb24db847 None admin] RBAC: Authorizing `identity:get_user(user_id=b1acb0a671c14b0fbc92babd2a80ad94)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:07:19.068494 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f00dda-059a-46ae-a782-765bb24db847 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:07:19.070065 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 578/1143] 10.222.0.61 () {56 vars in 1177 bytes} [Mon Aug 22 19:07:18 2022] GET /identity/v3/users/b1acb0a671c14b0fbc92babd2a80ad94 => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:07:19.076460 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011396f4-b021-4794-9d33-baac33255b1d None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:07:19.157616 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:07:19.175558 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011396f4-b021-4794-9d33-baac33255b1d None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:07:19.208662 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac875b0e-639a-4fb0-bb4a-29cb18e5178a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:07:19.209262 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac875b0e-639a-4fb0-bb4a-29cb18e5178a None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:19.209542 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac875b0e-639a-4fb0-bb4a-29cb18e5178a None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:19.209814 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac875b0e-639a-4fb0-bb4a-29cb18e5178a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:19.274980 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac875b0e-639a-4fb0-bb4a-29cb18e5178a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:07:19.276433 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac875b0e-639a-4fb0-bb4a-29cb18e5178a None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:07:19.353422 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:07:19.373342 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 566/1144] 10.222.0.61 () {58 vars in 1282 bytes} [Mon Aug 22 19:07:19 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 22 19:07:19.381111 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f00dda-059a-46ae-a782-765bb24db847 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:07:19.445311 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:07:19.477199 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f00dda-059a-46ae-a782-765bb24db847 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:07:19.511303 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a9700b-4e23-42f4-a49c-732a572fbc73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:07:19.511960 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a9700b-4e23-42f4-a49c-732a572fbc73 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:19.512234 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a9700b-4e23-42f4-a49c-732a572fbc73 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:19.512502 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a9700b-4e23-42f4-a49c-732a572fbc73 None admin] PATH_INFO: `/v3/projects/d5cade3019a742eda75f198a7cac3913` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:19.513861 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a9700b-4e23-42f4-a49c-732a572fbc73 None admin] RBAC: Authorizing `identity:get_project(project_id=d5cade3019a742eda75f198a7cac3913)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:07:19.515374 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a9700b-4e23-42f4-a49c-732a572fbc73 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:07:19.517028 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 579/1145] 10.222.0.61 () {56 vars in 1186 bytes} [Mon Aug 22 19:07:19 2022] GET /identity/v3/projects/d5cade3019a742eda75f198a7cac3913 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:07:19.523128 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac875b0e-639a-4fb0-bb4a-29cb18e5178a None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:07:19.619169 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac875b0e-639a-4fb0-bb4a-29cb18e5178a None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:07:19.651761 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa47842-cb48-4051-abb2-1a1814e7dd93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:07:19.652352 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa47842-cb48-4051-abb2-1a1814e7dd93 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:19.652629 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa47842-cb48-4051-abb2-1a1814e7dd93 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:19.652898 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa47842-cb48-4051-abb2-1a1814e7dd93 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:19.653899 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa47842-cb48-4051-abb2-1a1814e7dd93 None admin] RBAC: Adding query filter params (role.id=163c30c3b3c246beb849ce176f02d9da, scope.project.id=d5cade3019a742eda75f198a7cac3913, user.id=b1acb0a671c14b0fbc92babd2a80ad94) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:07:19.654018 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa47842-cb48-4051-abb2-1a1814e7dd93 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:07:19.655334 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa47842-cb48-4051-abb2-1a1814e7dd93 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:07:19.660589 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 567/1146] 10.222.0.61 () {56 vars in 1374 bytes} [Mon Aug 22 19:07:19 2022] GET /identity/v3/role_assignments?user.id=b1acb0a671c14b0fbc92babd2a80ad94&scope.project.id=d5cade3019a742eda75f198a7cac3913&role.id=163c30c3b3c246beb849ce176f02d9da => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:07:21.503230 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b8541d-492a-4963-b658-e1046ce856be None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:21.503904 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b8541d-492a-4963-b658-e1046ce856be None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:21.503904 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b8541d-492a-4963-b658-e1046ce856be None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:21.505177 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 580/1147] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:07:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:07:21.510133 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c0b483-7f49-4467-860c-d43e8f44f97c None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:21.510554 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c0b483-7f49-4467-860c-d43e8f44f97c None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:21.510677 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c0b483-7f49-4467-860c-d43e8f44f97c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:21.529170 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-00c0b483-7f49-4467-860c-d43e8f44f97c None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:07:21.530359 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-00c0b483-7f49-4467-860c-d43e8f44f97c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:07:21.571763 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:07:21.596289 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 568/1148] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:07:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 19:07:21.861165 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b8541d-492a-4963-b658-e1046ce856be None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:07:21.960609 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b8541d-492a-4963-b658-e1046ce856be None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:07:21.994044 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ca1ee4-b1f8-433f-b255-074de3678b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:07:21.994663 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ca1ee4-b1f8-433f-b255-074de3678b65 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:21.994941 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ca1ee4-b1f8-433f-b255-074de3678b65 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:21.995217 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ca1ee4-b1f8-433f-b255-074de3678b65 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:21.997833 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:07:22.005723 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ca1ee4-b1f8-433f-b255-074de3678b65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 19:07:22.010423 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-43ca1ee4-b1f8-433f-b255-074de3678b65 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 19:07:22.010423 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:07:22.010423 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:07:22.010423 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:07:22.010423 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:07:22.010423 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:07:22.010423 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:07:22.010423 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:07:22.010423 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:07:22.010423 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:07:22.010423 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:07:22.010423 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:07:22.010423 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 19:07:22.010423 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 19:07:22.010423 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 19:07:22.010423 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 19:07:22.011608 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 19:07:22.011608 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 19:07:22.011608 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 19:07:22.011608 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 19:07:22.011608 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:07:22.011608 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:07:22.011608 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:07:22.011608 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:07:22.011608 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:07:22.011608 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:07:22.011608 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:07:22.011608 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:07:22.011608 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:07:22.011608 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:07:22.011608 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:07:22.011608 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:07:22.012861 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:07:22.012861 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:07:22.012861 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:07:22.012861 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:07:22.012861 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 19:07:22.012861 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 19:07:22.012861 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 19:07:22.012861 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 19:07:22.012861 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 19:07:22.012861 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 19:07:22.012861 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 19:07:22.012861 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:07:22.012861 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 581/1149] 10.222.0.61 () {56 vars in 1096 bytes} [Mon Aug 22 19:07:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:07:22.017656 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c0b483-7f49-4467-860c-d43e8f44f97c None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:07:22.113767 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c0b483-7f49-4467-860c-d43e8f44f97c None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:07:22.146403 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a74574c-c01a-45ea-ae45-9ca51ffdd350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:07:22.147014 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a74574c-c01a-45ea-ae45-9ca51ffdd350 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:22.147289 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a74574c-c01a-45ea-ae45-9ca51ffdd350 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:22.147557 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a74574c-c01a-45ea-ae45-9ca51ffdd350 None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:22.148261 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a74574c-c01a-45ea-ae45-9ca51ffdd350 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:07:22.148557 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a74574c-c01a-45ea-ae45-9ca51ffdd350 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:07:22.149909 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a74574c-c01a-45ea-ae45-9ca51ffdd350 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:07:22.158496 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 569/1150] 10.222.0.61 () {56 vars in 1099 bytes} [Mon Aug 22 19:07:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:07:22.164277 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ca1ee4-b1f8-433f-b255-074de3678b65 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:07:22.263492 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ca1ee4-b1f8-433f-b255-074de3678b65 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:07:22.297372 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4223c87-78ee-45f5-ab28-2b6b57d31de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:07:22.297991 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4223c87-78ee-45f5-ab28-2b6b57d31de2 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:22.298264 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4223c87-78ee-45f5-ab28-2b6b57d31de2 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:22.298533 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4223c87-78ee-45f5-ab28-2b6b57d31de2 None admin] PATH_INFO: `/v3/users/b1acb0a671c14b0fbc92babd2a80ad94` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:22.299630 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4223c87-78ee-45f5-ab28-2b6b57d31de2 None admin] RBAC: Authorizing `identity:get_user(user_id=b1acb0a671c14b0fbc92babd2a80ad94)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:07:22.301070 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4223c87-78ee-45f5-ab28-2b6b57d31de2 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:07:22.302625 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 582/1151] 10.222.0.61 () {56 vars in 1177 bytes} [Mon Aug 22 19:07:22 2022] GET /identity/v3/users/b1acb0a671c14b0fbc92babd2a80ad94 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:07:22.308420 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a74574c-c01a-45ea-ae45-9ca51ffdd350 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:07:22.348515 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:07:22.409423 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a74574c-c01a-45ea-ae45-9ca51ffdd350 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:07:22.442108 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1f7863-1414-4862-a1d3-016ca29ab8c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:07:22.442732 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1f7863-1414-4862-a1d3-016ca29ab8c7 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:22.443011 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1f7863-1414-4862-a1d3-016ca29ab8c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:22.443283 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1f7863-1414-4862-a1d3-016ca29ab8c7 None admin] PATH_INFO: `/v3/projects/d5cade3019a742eda75f198a7cac3913` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:22.444430 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1f7863-1414-4862-a1d3-016ca29ab8c7 None admin] RBAC: Authorizing `identity:get_project(project_id=d5cade3019a742eda75f198a7cac3913)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:07:22.445848 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1f7863-1414-4862-a1d3-016ca29ab8c7 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:07:22.447483 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 570/1152] 10.222.0.61 () {56 vars in 1186 bytes} [Mon Aug 22 19:07:22 2022] GET /identity/v3/projects/d5cade3019a742eda75f198a7cac3913 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:07:22.453911 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4223c87-78ee-45f5-ab28-2b6b57d31de2 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:07:22.515780 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:07:22.552287 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4223c87-78ee-45f5-ab28-2b6b57d31de2 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:07:22.585008 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f5a00f-0638-4722-b2ad-a7617f0976b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:07:22.585646 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f5a00f-0638-4722-b2ad-a7617f0976b6 None admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:22.585923 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f5a00f-0638-4722-b2ad-a7617f0976b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:22.586195 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f5a00f-0638-4722-b2ad-a7617f0976b6 None admin] PATH_INFO: `/v3/projects/d5cade3019a742eda75f198a7cac3913/users/b1acb0a671c14b0fbc92babd2a80ad94/roles/163c30c3b3c246beb849ce176f02d9da` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:22.588105 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f5a00f-0638-4722-b2ad-a7617f0976b6 None admin] RBAC: Authorizing `identity:create_grant(project_id=d5cade3019a742eda75f198a7cac3913, user_id=b1acb0a671c14b0fbc92babd2a80ad94, role_id=163c30c3b3c246beb849ce176f02d9da)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:07:22.589513 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f5a00f-0638-4722-b2ad-a7617f0976b6 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:07:22.602870 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 583/1153] 10.222.0.61 () {58 vars in 1439 bytes} [Mon Aug 22 19:07:22 2022] PUT /identity/v3/projects/d5cade3019a742eda75f198a7cac3913/users/b1acb0a671c14b0fbc92babd2a80ad94/roles/163c30c3b3c246beb849ce176f02d9da => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:07:24.453899 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950acccf-8b44-4dc7-a6ec-6143e9067693 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:24.454454 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950acccf-8b44-4dc7-a6ec-6143e9067693 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:24.454454 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950acccf-8b44-4dc7-a6ec-6143e9067693 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:24.455813 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 571/1154] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:07:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:07:24.461127 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f3383d-eaa9-486f-a44c-abde598c8717 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:24.461558 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f3383d-eaa9-486f-a44c-abde598c8717 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:24.461674 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f3383d-eaa9-486f-a44c-abde598c8717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:24.479266 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-74f3383d-eaa9-486f-a44c-abde598c8717 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:07:24.480485 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-74f3383d-eaa9-486f-a44c-abde598c8717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:07:24.545226 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 584/1155] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:07:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 19:07:24.809483 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950acccf-8b44-4dc7-a6ec-6143e9067693 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:07:24.907045 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950acccf-8b44-4dc7-a6ec-6143e9067693 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:07:24.913958 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:07:24.940569 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563fcad2-bb03-42da-b289-1377a832ba27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:07:24.941181 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563fcad2-bb03-42da-b289-1377a832ba27 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:24.941456 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563fcad2-bb03-42da-b289-1377a832ba27 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:24.941720 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563fcad2-bb03-42da-b289-1377a832ba27 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:24.951333 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563fcad2-bb03-42da-b289-1377a832ba27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 19:07:24.955966 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-563fcad2-bb03-42da-b289-1377a832ba27 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 19:07:24.955966 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:07:24.955966 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:07:24.955966 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:07:24.955966 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:07:24.955966 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:07:24.955966 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:07:24.955966 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:07:24.955966 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:07:24.955966 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:07:24.955966 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:07:24.955966 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:07:24.955966 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 19:07:24.955966 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 19:07:24.955966 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 19:07:24.955966 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 19:07:24.957062 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 19:07:24.957062 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 19:07:24.957062 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 19:07:24.957062 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 19:07:24.957062 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:07:24.957062 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:07:24.957062 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:07:24.957062 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:07:24.957062 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:07:24.957062 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:07:24.957062 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:07:24.957062 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:07:24.957062 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:07:24.957062 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:07:24.957062 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:07:24.957062 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:07:24.958105 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:07:24.958105 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:07:24.958105 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:07:24.958105 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:07:24.958105 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 19:07:24.958105 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 19:07:24.958105 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 19:07:24.958105 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 19:07:24.958105 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 19:07:24.958105 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 19:07:24.958105 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 19:07:24.958105 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:07:24.958105 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 572/1156] 10.222.0.61 () {56 vars in 1096 bytes} [Mon Aug 22 19:07:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:07:24.963032 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f3383d-eaa9-486f-a44c-abde598c8717 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:07:25.017307 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:07:25.061611 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f3383d-eaa9-486f-a44c-abde598c8717 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:07:25.094198 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d7cbea-4ce5-426f-a1c8-d046643e3b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:07:25.094795 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d7cbea-4ce5-426f-a1c8-d046643e3b66 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:25.095071 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d7cbea-4ce5-426f-a1c8-d046643e3b66 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:25.095342 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d7cbea-4ce5-426f-a1c8-d046643e3b66 None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:25.096033 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d7cbea-4ce5-426f-a1c8-d046643e3b66 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:07:25.096333 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d7cbea-4ce5-426f-a1c8-d046643e3b66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:07:25.097691 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d7cbea-4ce5-426f-a1c8-d046643e3b66 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:07:25.105456 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 585/1157] 10.222.0.61 () {56 vars in 1099 bytes} [Mon Aug 22 19:07:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:07:25.111870 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563fcad2-bb03-42da-b289-1377a832ba27 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:07:25.212855 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563fcad2-bb03-42da-b289-1377a832ba27 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:07:25.246438 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948a3b59-05ec-4f6c-9016-c38a811d49a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:07:25.247058 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948a3b59-05ec-4f6c-9016-c38a811d49a2 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:25.247365 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948a3b59-05ec-4f6c-9016-c38a811d49a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:25.247659 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948a3b59-05ec-4f6c-9016-c38a811d49a2 None admin] PATH_INFO: `/v3/users/b1acb0a671c14b0fbc92babd2a80ad94` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:25.248791 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948a3b59-05ec-4f6c-9016-c38a811d49a2 None admin] RBAC: Authorizing `identity:get_user(user_id=b1acb0a671c14b0fbc92babd2a80ad94)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:07:25.250208 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948a3b59-05ec-4f6c-9016-c38a811d49a2 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:07:25.251811 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 573/1158] 10.222.0.61 () {56 vars in 1177 bytes} [Mon Aug 22 19:07:25 2022] GET /identity/v3/users/b1acb0a671c14b0fbc92babd2a80ad94 => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:07:25.257707 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d7cbea-4ce5-426f-a1c8-d046643e3b66 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:07:25.355583 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d7cbea-4ce5-426f-a1c8-d046643e3b66 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:07:25.371184 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:07:25.389557 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3741e4-85a1-49a6-ab94-067ec08860d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:07:25.390155 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3741e4-85a1-49a6-ab94-067ec08860d5 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:25.390435 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3741e4-85a1-49a6-ab94-067ec08860d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:25.390707 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3741e4-85a1-49a6-ab94-067ec08860d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:25.456287 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3741e4-85a1-49a6-ab94-067ec08860d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:07:25.457721 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3741e4-85a1-49a6-ab94-067ec08860d5 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:07:25.554597 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 586/1159] 10.222.0.61 () {58 vars in 1282 bytes} [Mon Aug 22 19:07:25 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 22 19:07:25.560545 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948a3b59-05ec-4f6c-9016-c38a811d49a2 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:07:25.594367 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:07:25.660263 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948a3b59-05ec-4f6c-9016-c38a811d49a2 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:07:25.693502 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790b3d5f-a0b2-45e1-8742-70bbb56f131b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:07:25.694121 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790b3d5f-a0b2-45e1-8742-70bbb56f131b None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:25.694401 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790b3d5f-a0b2-45e1-8742-70bbb56f131b None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:25.694671 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790b3d5f-a0b2-45e1-8742-70bbb56f131b None admin] PATH_INFO: `/v3/projects/d5cade3019a742eda75f198a7cac3913` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:25.695813 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790b3d5f-a0b2-45e1-8742-70bbb56f131b None admin] RBAC: Authorizing `identity:get_project(project_id=d5cade3019a742eda75f198a7cac3913)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:07:25.697223 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790b3d5f-a0b2-45e1-8742-70bbb56f131b None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:07:25.698843 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 574/1160] 10.222.0.61 () {56 vars in 1186 bytes} [Mon Aug 22 19:07:25 2022] GET /identity/v3/projects/d5cade3019a742eda75f198a7cac3913 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:07:25.705539 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3741e4-85a1-49a6-ab94-067ec08860d5 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:07:25.723053 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:07:25.803058 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3741e4-85a1-49a6-ab94-067ec08860d5 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:07:25.836344 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68429993-96d1-4d53-b239-db0d88286441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:07:25.836938 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68429993-96d1-4d53-b239-db0d88286441 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:25.837221 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68429993-96d1-4d53-b239-db0d88286441 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:25.837493 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68429993-96d1-4d53-b239-db0d88286441 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:25.838211 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68429993-96d1-4d53-b239-db0d88286441 None admin] RBAC: Adding query filter params (role.id=163c30c3b3c246beb849ce176f02d9da, scope.project.id=d5cade3019a742eda75f198a7cac3913, user.id=b1acb0a671c14b0fbc92babd2a80ad94) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:07:25.838520 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68429993-96d1-4d53-b239-db0d88286441 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:07:25.839961 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68429993-96d1-4d53-b239-db0d88286441 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:07:25.845559 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 587/1161] 10.222.0.61 () {56 vars in 1374 bytes} [Mon Aug 22 19:07:25 2022] GET /identity/v3/role_assignments?user.id=b1acb0a671c14b0fbc92babd2a80ad94&scope.project.id=d5cade3019a742eda75f198a7cac3913&role.id=163c30c3b3c246beb849ce176f02d9da => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:07:27.727189 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b660ba72-69b6-4917-ab63-bf3fdd1fe2f6 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:27.727760 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b660ba72-69b6-4917-ab63-bf3fdd1fe2f6 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:27.727760 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b660ba72-69b6-4917-ab63-bf3fdd1fe2f6 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:27.729123 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 575/1162] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:07:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:07:27.734323 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58956af2-acb1-4637-87a5-c69a4f43f3d0 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:27.734715 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58956af2-acb1-4637-87a5-c69a4f43f3d0 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:27.734867 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58956af2-acb1-4637-87a5-c69a4f43f3d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:27.752684 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-58956af2-acb1-4637-87a5-c69a4f43f3d0 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:07:27.753859 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-58956af2-acb1-4637-87a5-c69a4f43f3d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:07:27.818087 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 588/1163] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:07:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 19:07:28.087802 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b660ba72-69b6-4917-ab63-bf3fdd1fe2f6 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:07:28.185570 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b660ba72-69b6-4917-ab63-bf3fdd1fe2f6 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:07:28.187052 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:07:28.219603 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4691c2-af97-4995-9b36-655a862414ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:07:28.220224 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4691c2-af97-4995-9b36-655a862414ba None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:28.220500 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4691c2-af97-4995-9b36-655a862414ba None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:28.220767 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4691c2-af97-4995-9b36-655a862414ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:28.285321 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4691c2-af97-4995-9b36-655a862414ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:07:28.286749 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4691c2-af97-4995-9b36-655a862414ba None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:07:28.381192 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 576/1164] 10.222.0.61 () {58 vars in 1282 bytes} [Mon Aug 22 19:07:28 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 22 19:07:28.387098 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58956af2-acb1-4637-87a5-c69a4f43f3d0 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:07:28.398407 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:07:28.482432 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58956af2-acb1-4637-87a5-c69a4f43f3d0 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:07:28.518341 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a7806d-6304-4f5f-b097-d34b227ed1d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:07:28.518972 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a7806d-6304-4f5f-b097-d34b227ed1d5 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:28.519251 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a7806d-6304-4f5f-b097-d34b227ed1d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:28.519519 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a7806d-6304-4f5f-b097-d34b227ed1d5 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:28.526019 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a7806d-6304-4f5f-b097-d34b227ed1d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:07:28.527441 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a7806d-6304-4f5f-b097-d34b227ed1d5 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:07:28.538722 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-88a7806d-6304-4f5f-b097-d34b227ed1d5 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 22 19:07:28.538722 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:07:28.538722 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 22 19:07:28.538722 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 22 19:07:28.538722 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 19:07:28.538722 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 19:07:28.538722 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 19:07:28.538722 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 19:07:28.538722 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 22 19:07:28.538722 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:07:28.538722 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 19:07:28.538722 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:07:28.538722 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:07:28.538722 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:07:28.538722 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:07:28.538722 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:07:28.538722 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:07:28.539963 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:07:28.539963 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:07:28.539963 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:07:28.539963 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:07:28.539963 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:07:28.539963 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:07:28.539963 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 19:07:28.539963 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 19:07:28.539963 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 19:07:28.539963 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 19:07:28.539963 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:07:28.539963 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:07:28.539963 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:07:28.539963 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:07:28.539963 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:07:28.539963 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:07:28.541020 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:07:28.541020 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:07:28.541020 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:07:28.541020 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:07:28.541020 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:07:28.541020 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:07:28.541020 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:07:28.541020 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:07:28.541020 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:07:28.541020 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:07:28.541020 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 22 19:07:28.541020 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 19:07:28.541020 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 22 19:07:28.541020 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:07:28.541020 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 589/1165] 10.222.0.61 () {56 vars in 1117 bytes} [Mon Aug 22 19:07:28 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:07:28.546476 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4691c2-af97-4995-9b36-655a862414ba None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:07:28.548683 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:07:28.642650 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4691c2-af97-4995-9b36-655a862414ba None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:07:28.674944 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0c6f4d-943f-4589-a2a0-e9f1f125d737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:07:28.675537 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0c6f4d-943f-4589-a2a0-e9f1f125d737 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:28.675838 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0c6f4d-943f-4589-a2a0-e9f1f125d737 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:28.676103 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0c6f4d-943f-4589-a2a0-e9f1f125d737 None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:28.676729 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0c6f4d-943f-4589-a2a0-e9f1f125d737 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:07:28.677024 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0c6f4d-943f-4589-a2a0-e9f1f125d737 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:07:28.678360 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0c6f4d-943f-4589-a2a0-e9f1f125d737 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:07:28.685817 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 577/1166] 10.222.0.61 () {56 vars in 1115 bytes} [Mon Aug 22 19:07:28 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:07:30.557871 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79e5631-5412-4a5f-93d2-4faf408047e0 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:30.558424 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79e5631-5412-4a5f-93d2-4faf408047e0 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:30.558424 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79e5631-5412-4a5f-93d2-4faf408047e0 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:30.559806 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 590/1167] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:07:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:07:30.564945 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33470669-5c7f-4525-bb0d-d0b3aea9ea67 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:30.565392 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33470669-5c7f-4525-bb0d-d0b3aea9ea67 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:30.565486 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33470669-5c7f-4525-bb0d-d0b3aea9ea67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:30.583527 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-33470669-5c7f-4525-bb0d-d0b3aea9ea67 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:07:30.584746 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-33470669-5c7f-4525-bb0d-d0b3aea9ea67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:07:30.641840 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:07:30.649789 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 578/1168] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:07:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 19:07:30.919009 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79e5631-5412-4a5f-93d2-4faf408047e0 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:07:30.986683 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:07:31.018452 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79e5631-5412-4a5f-93d2-4faf408047e0 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:07:31.050013 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be85001a-5f74-470f-9c33-01028440941b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:07:31.050601 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be85001a-5f74-470f-9c33-01028440941b None admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:31.050874 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be85001a-5f74-470f-9c33-01028440941b None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:31.051141 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be85001a-5f74-470f-9c33-01028440941b None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:31.051819 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be85001a-5f74-470f-9c33-01028440941b None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:07:31.053174 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be85001a-5f74-470f-9c33-01028440941b None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:07:31.068237 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 591/1169] 10.222.0.61 () {60 vars in 1138 bytes} [Mon Aug 22 19:07:30 2022] POST /identity/v3/domains => generated 267 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:07:32.952593 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8b2c82-8add-4903-bc20-bc254cf03fb8 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:32.953190 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8b2c82-8add-4903-bc20-bc254cf03fb8 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:32.953190 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8b2c82-8add-4903-bc20-bc254cf03fb8 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:32.954486 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 579/1170] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:07:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:07:32.959731 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04e25bc-203d-473e-aadf-77bec0d3fc87 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:32.960131 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04e25bc-203d-473e-aadf-77bec0d3fc87 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:32.960272 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04e25bc-203d-473e-aadf-77bec0d3fc87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:32.977768 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-c04e25bc-203d-473e-aadf-77bec0d3fc87 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:07:32.978941 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-c04e25bc-203d-473e-aadf-77bec0d3fc87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:07:33.042851 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 592/1171] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:07:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 19:07:33.311784 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8b2c82-8add-4903-bc20-bc254cf03fb8 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:07:33.409912 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8b2c82-8add-4903-bc20-bc254cf03fb8 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:07:33.445251 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239a00b6-e51f-406b-84f6-84e7aaac183f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:07:33.445875 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239a00b6-e51f-406b-84f6-84e7aaac183f None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:33.446155 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239a00b6-e51f-406b-84f6-84e7aaac183f None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:33.446430 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239a00b6-e51f-406b-84f6-84e7aaac183f None admin] PATH_INFO: `/v3/domains/ae8a1a83d2194880bd447aaa3a1f045d` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:33.457051 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239a00b6-e51f-406b-84f6-84e7aaac183f None admin] RBAC: Authorizing `identity:get_domain(domain_id=ae8a1a83d2194880bd447aaa3a1f045d)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:07:33.458467 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239a00b6-e51f-406b-84f6-84e7aaac183f None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:07:33.460070 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 580/1172] 10.222.0.61 () {56 vars in 1183 bytes} [Mon Aug 22 19:07:33 2022] GET /identity/v3/domains/ae8a1a83d2194880bd447aaa3a1f045d => generated 267 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:07:33.465866 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04e25bc-203d-473e-aadf-77bec0d3fc87 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:07:33.523543 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:07:33.563331 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04e25bc-203d-473e-aadf-77bec0d3fc87 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:07:33.597450 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cdc18e-37f0-42f9-ab10-77f7e819ea6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:07:33.598054 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cdc18e-37f0-42f9-ab10-77f7e819ea6c None admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:33.598332 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cdc18e-37f0-42f9-ab10-77f7e819ea6c None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:33.598604 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cdc18e-37f0-42f9-ab10-77f7e819ea6c None admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:33.599241 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cdc18e-37f0-42f9-ab10-77f7e819ea6c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:07:33.600703 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cdc18e-37f0-42f9-ab10-77f7e819ea6c None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:07:33.626745 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 593/1173] 10.222.0.61 () {60 vars in 1141 bytes} [Mon Aug 22 19:07:33 2022] POST /identity/v3/projects => generated 361 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:07:35.504839 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fd47d5-f1a4-4741-9c8b-5e0386ed3b97 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:35.505363 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fd47d5-f1a4-4741-9c8b-5e0386ed3b97 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:35.505363 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fd47d5-f1a4-4741-9c8b-5e0386ed3b97 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:35.506756 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 581/1174] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:07:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:07:35.511867 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7785086d-fa9e-486b-acc4-41cdadfaa64d None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:35.512285 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7785086d-fa9e-486b-acc4-41cdadfaa64d None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:35.512456 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7785086d-fa9e-486b-acc4-41cdadfaa64d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:35.530627 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-7785086d-fa9e-486b-acc4-41cdadfaa64d None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:07:35.531828 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-7785086d-fa9e-486b-acc4-41cdadfaa64d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:07:35.595473 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 594/1175] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:07:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 19:07:35.859771 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fd47d5-f1a4-4741-9c8b-5e0386ed3b97 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:07:35.915711 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:07:35.959389 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fd47d5-f1a4-4741-9c8b-5e0386ed3b97 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:07:35.992146 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc34ebb-a0dc-4b8a-91ce-ff08b4260c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:07:35.992763 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc34ebb-a0dc-4b8a-91ce-ff08b4260c11 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:35.993040 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc34ebb-a0dc-4b8a-91ce-ff08b4260c11 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:35.993310 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc34ebb-a0dc-4b8a-91ce-ff08b4260c11 None admin] PATH_INFO: `/v3/domains/ae8a1a83d2194880bd447aaa3a1f045d` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:35.994418 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc34ebb-a0dc-4b8a-91ce-ff08b4260c11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=ae8a1a83d2194880bd447aaa3a1f045d)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:07:35.995859 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc34ebb-a0dc-4b8a-91ce-ff08b4260c11 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:07:35.998049 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 582/1176] 10.222.0.61 () {56 vars in 1183 bytes} [Mon Aug 22 19:07:35 2022] GET /identity/v3/domains/ae8a1a83d2194880bd447aaa3a1f045d => generated 267 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:07:36.003577 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7785086d-fa9e-486b-acc4-41cdadfaa64d None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:07:36.044671 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:07:36.100519 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7785086d-fa9e-486b-acc4-41cdadfaa64d None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:07:36.134844 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f57d7b-0139-48c3-a3a5-7828fa7076e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:07:36.135387 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f57d7b-0139-48c3-a3a5-7828fa7076e2 None admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:36.135688 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f57d7b-0139-48c3-a3a5-7828fa7076e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:36.135966 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f57d7b-0139-48c3-a3a5-7828fa7076e2 None admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:36.136601 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f57d7b-0139-48c3-a3a5-7828fa7076e2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:07:36.138017 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f57d7b-0139-48c3-a3a5-7828fa7076e2 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:07:36.158566 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 595/1177] 10.222.0.61 () {60 vars in 1132 bytes} [Mon Aug 22 19:07:36 2022] POST /identity/v3/users => generated 311 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:07:38.055336 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350f9783-723e-4090-a1a1-90110a9a7462 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:38.055912 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350f9783-723e-4090-a1a1-90110a9a7462 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:38.055912 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350f9783-723e-4090-a1a1-90110a9a7462 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:38.057290 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 583/1178] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:07:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:07:38.062492 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba282b4d-c721-4633-a3d0-3b0835a114b4 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:38.062927 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba282b4d-c721-4633-a3d0-3b0835a114b4 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:38.063020 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba282b4d-c721-4633-a3d0-3b0835a114b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:38.081917 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-ba282b4d-c721-4633-a3d0-3b0835a114b4 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:07:38.083095 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-ba282b4d-c721-4633-a3d0-3b0835a114b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:07:38.147894 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 596/1179] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:07:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 19:07:38.417363 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350f9783-723e-4090-a1a1-90110a9a7462 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:07:38.515404 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350f9783-723e-4090-a1a1-90110a9a7462 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:07:38.537638 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:07:38.548148 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1566b0-7e37-4279-bfd4-327ed0df4b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:07:38.548768 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1566b0-7e37-4279-bfd4-327ed0df4b84 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:38.549044 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1566b0-7e37-4279-bfd4-327ed0df4b84 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:38.549313 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1566b0-7e37-4279-bfd4-327ed0df4b84 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:38.559013 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1566b0-7e37-4279-bfd4-327ed0df4b84 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 19:07:38.564167 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-bb1566b0-7e37-4279-bfd4-327ed0df4b84 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 19:07:38.564167 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:07:38.564167 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:07:38.564167 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:07:38.564167 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:07:38.564167 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:07:38.564167 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:07:38.564167 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:07:38.564167 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:07:38.564167 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:07:38.564167 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:07:38.564167 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:07:38.564167 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 19:07:38.564167 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 19:07:38.564167 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 19:07:38.564167 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 19:07:38.565145 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 19:07:38.565145 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 19:07:38.565145 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 19:07:38.565145 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 19:07:38.565145 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:07:38.565145 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:07:38.565145 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:07:38.565145 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:07:38.565145 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:07:38.565145 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:07:38.565145 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:07:38.565145 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:07:38.565145 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:07:38.565145 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:07:38.565145 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:07:38.565145 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:07:38.566067 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:07:38.566067 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:07:38.566067 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:07:38.566067 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:07:38.566067 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 19:07:38.566067 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 19:07:38.566067 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 19:07:38.566067 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 19:07:38.566067 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 19:07:38.566067 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 19:07:38.566067 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 19:07:38.566067 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:07:38.566067 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 584/1180] 10.222.0.61 () {56 vars in 1096 bytes} [Mon Aug 22 19:07:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:07:38.571095 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba282b4d-c721-4633-a3d0-3b0835a114b4 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:07:38.597864 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:07:38.672117 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba282b4d-c721-4633-a3d0-3b0835a114b4 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:07:38.705491 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b13e61-e4f7-4378-8a6b-dcf575960c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:07:38.706097 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b13e61-e4f7-4378-8a6b-dcf575960c66 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:38.706373 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b13e61-e4f7-4378-8a6b-dcf575960c66 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:38.706640 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b13e61-e4f7-4378-8a6b-dcf575960c66 None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:38.707300 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b13e61-e4f7-4378-8a6b-dcf575960c66 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:07:38.707601 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b13e61-e4f7-4378-8a6b-dcf575960c66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:07:38.708987 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b13e61-e4f7-4378-8a6b-dcf575960c66 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:07:38.716767 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 597/1181] 10.222.0.61 () {56 vars in 1099 bytes} [Mon Aug 22 19:07:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:07:38.722497 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1566b0-7e37-4279-bfd4-327ed0df4b84 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:07:38.818801 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1566b0-7e37-4279-bfd4-327ed0df4b84 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:07:38.851225 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c86ead-bb01-4553-8822-dbc0faca7441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:07:38.851857 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c86ead-bb01-4553-8822-dbc0faca7441 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:38.852132 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c86ead-bb01-4553-8822-dbc0faca7441 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:38.852397 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c86ead-bb01-4553-8822-dbc0faca7441 None admin] PATH_INFO: `/v3/users/17a182d20c5947858a5a5d53aeaadc5d` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:38.865518 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c86ead-bb01-4553-8822-dbc0faca7441 None admin] RBAC: Authorizing `identity:get_user(user_id=17a182d20c5947858a5a5d53aeaadc5d)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:07:38.866922 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c86ead-bb01-4553-8822-dbc0faca7441 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:07:38.868514 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 585/1182] 10.222.0.61 () {56 vars in 1177 bytes} [Mon Aug 22 19:07:38 2022] GET /identity/v3/users/17a182d20c5947858a5a5d53aeaadc5d => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:07:38.874773 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b13e61-e4f7-4378-8a6b-dcf575960c66 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:07:38.961645 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:07:38.971140 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b13e61-e4f7-4378-8a6b-dcf575960c66 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:07:39.004229 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d61aef4-3165-4c16-a11a-2ff38e6239d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:07:39.004825 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d61aef4-3165-4c16-a11a-2ff38e6239d3 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:39.005098 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d61aef4-3165-4c16-a11a-2ff38e6239d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:39.005364 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d61aef4-3165-4c16-a11a-2ff38e6239d3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:39.069435 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d61aef4-3165-4c16-a11a-2ff38e6239d3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:07:39.070862 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d61aef4-3165-4c16-a11a-2ff38e6239d3 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:07:39.156100 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:07:39.166383 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 598/1183] 10.222.0.61 () {58 vars in 1282 bytes} [Mon Aug 22 19:07:38 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 22 19:07:39.172747 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c86ead-bb01-4553-8822-dbc0faca7441 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:07:39.214979 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:07:39.270071 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c86ead-bb01-4553-8822-dbc0faca7441 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:07:39.303242 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58017966-b0d5-4583-b3b3-e553d0d0a7a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:07:39.303887 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58017966-b0d5-4583-b3b3-e553d0d0a7a2 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:39.304168 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58017966-b0d5-4583-b3b3-e553d0d0a7a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:39.304442 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58017966-b0d5-4583-b3b3-e553d0d0a7a2 None admin] PATH_INFO: `/v3/projects/b2bd4ad0f0dd4069ba3336740f72da62` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:39.305549 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58017966-b0d5-4583-b3b3-e553d0d0a7a2 None admin] RBAC: Authorizing `identity:get_project(project_id=b2bd4ad0f0dd4069ba3336740f72da62)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:07:39.306958 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58017966-b0d5-4583-b3b3-e553d0d0a7a2 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:07:39.308595 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 586/1184] 10.222.0.61 () {56 vars in 1186 bytes} [Mon Aug 22 19:07:39 2022] GET /identity/v3/projects/b2bd4ad0f0dd4069ba3336740f72da62 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:07:39.314466 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d61aef4-3165-4c16-a11a-2ff38e6239d3 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:07:39.410375 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d61aef4-3165-4c16-a11a-2ff38e6239d3 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:07:39.442949 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96451ea5-868c-4fe5-acd0-386388ea200d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:07:39.443537 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96451ea5-868c-4fe5-acd0-386388ea200d None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:39.443838 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96451ea5-868c-4fe5-acd0-386388ea200d None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:39.444106 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96451ea5-868c-4fe5-acd0-386388ea200d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:39.444814 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96451ea5-868c-4fe5-acd0-386388ea200d None admin] RBAC: Adding query filter params (role.id=163c30c3b3c246beb849ce176f02d9da, scope.project.id=b2bd4ad0f0dd4069ba3336740f72da62, user.id=17a182d20c5947858a5a5d53aeaadc5d) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:07:39.445112 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96451ea5-868c-4fe5-acd0-386388ea200d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:07:39.446505 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96451ea5-868c-4fe5-acd0-386388ea200d None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:07:39.451699 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 599/1185] 10.222.0.61 () {56 vars in 1374 bytes} [Mon Aug 22 19:07:39 2022] GET /identity/v3/role_assignments?user.id=17a182d20c5947858a5a5d53aeaadc5d&scope.project.id=b2bd4ad0f0dd4069ba3336740f72da62&role.id=163c30c3b3c246beb849ce176f02d9da => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:07:41.306310 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aae5cbb-4d4b-4703-89ef-2208184249dd None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:41.306892 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aae5cbb-4d4b-4703-89ef-2208184249dd None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:41.306892 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aae5cbb-4d4b-4703-89ef-2208184249dd None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:41.308218 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 587/1186] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:07:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:07:41.313236 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ccbfaf-60ea-4edc-b2bd-d5bf6f5a7f34 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:41.313665 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ccbfaf-60ea-4edc-b2bd-d5bf6f5a7f34 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:41.313773 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ccbfaf-60ea-4edc-b2bd-d5bf6f5a7f34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:41.332084 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-40ccbfaf-60ea-4edc-b2bd-d5bf6f5a7f34 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:07:41.333281 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-40ccbfaf-60ea-4edc-b2bd-d5bf6f5a7f34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:07:41.382294 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:07:41.400270 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 600/1187] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:07:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 19:07:41.663807 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aae5cbb-4d4b-4703-89ef-2208184249dd None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:07:41.764025 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aae5cbb-4d4b-4703-89ef-2208184249dd None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:07:41.774783 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:07:41.797399 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90636184-2707-4be3-822e-2d628c899cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:07:41.798015 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90636184-2707-4be3-822e-2d628c899cb6 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:41.798286 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90636184-2707-4be3-822e-2d628c899cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:41.798552 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90636184-2707-4be3-822e-2d628c899cb6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:41.808173 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90636184-2707-4be3-822e-2d628c899cb6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 19:07:41.813303 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-90636184-2707-4be3-822e-2d628c899cb6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 19:07:41.813303 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:07:41.813303 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:07:41.813303 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:07:41.813303 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:07:41.813303 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:07:41.813303 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:07:41.813303 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:07:41.813303 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:07:41.813303 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:07:41.813303 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:07:41.813303 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:07:41.813303 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 19:07:41.813303 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 19:07:41.813303 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 19:07:41.813303 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 19:07:41.814467 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 19:07:41.814467 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 19:07:41.814467 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 19:07:41.814467 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 19:07:41.814467 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:07:41.814467 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:07:41.814467 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:07:41.814467 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:07:41.814467 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:07:41.814467 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:07:41.814467 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:07:41.814467 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:07:41.814467 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:07:41.814467 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:07:41.814467 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:07:41.814467 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:07:41.815413 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:07:41.815413 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:07:41.815413 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:07:41.815413 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:07:41.815413 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 19:07:41.815413 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 19:07:41.815413 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 19:07:41.815413 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 19:07:41.815413 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 19:07:41.815413 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 19:07:41.815413 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 19:07:41.815413 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:07:41.815413 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 588/1188] 10.222.0.61 () {56 vars in 1096 bytes} [Mon Aug 22 19:07:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:07:41.820469 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ccbfaf-60ea-4edc-b2bd-d5bf6f5a7f34 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:07:41.915811 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ccbfaf-60ea-4edc-b2bd-d5bf6f5a7f34 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:07:41.948855 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cede7f9-f160-4a1a-98bd-4393429ddb5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:07:41.949447 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cede7f9-f160-4a1a-98bd-4393429ddb5b None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:41.949751 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cede7f9-f160-4a1a-98bd-4393429ddb5b None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:41.950030 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cede7f9-f160-4a1a-98bd-4393429ddb5b None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:41.950686 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cede7f9-f160-4a1a-98bd-4393429ddb5b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:07:41.950987 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cede7f9-f160-4a1a-98bd-4393429ddb5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:07:41.952378 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cede7f9-f160-4a1a-98bd-4393429ddb5b None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:07:41.960104 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 601/1189] 10.222.0.61 () {56 vars in 1099 bytes} [Mon Aug 22 19:07:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:07:41.965762 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90636184-2707-4be3-822e-2d628c899cb6 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:07:42.067196 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90636184-2707-4be3-822e-2d628c899cb6 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:07:42.099076 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c793afc-1fb7-4b2c-a0f6-175a39c7ee92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:07:42.099715 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c793afc-1fb7-4b2c-a0f6-175a39c7ee92 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:42.099997 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c793afc-1fb7-4b2c-a0f6-175a39c7ee92 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:42.100271 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c793afc-1fb7-4b2c-a0f6-175a39c7ee92 None admin] PATH_INFO: `/v3/users/17a182d20c5947858a5a5d53aeaadc5d` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:42.101386 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c793afc-1fb7-4b2c-a0f6-175a39c7ee92 None admin] RBAC: Authorizing `identity:get_user(user_id=17a182d20c5947858a5a5d53aeaadc5d)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:07:42.102810 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c793afc-1fb7-4b2c-a0f6-175a39c7ee92 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:07:42.104399 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 589/1190] 10.222.0.61 () {56 vars in 1177 bytes} [Mon Aug 22 19:07:41 2022] GET /identity/v3/users/17a182d20c5947858a5a5d53aeaadc5d => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:07:42.110463 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cede7f9-f160-4a1a-98bd-4393429ddb5b None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:07:42.156453 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:07:42.208023 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cede7f9-f160-4a1a-98bd-4393429ddb5b None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:07:42.240715 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5978e54a-8766-4def-876e-91284b627cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:07:42.241341 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5978e54a-8766-4def-876e-91284b627cda None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:42.241618 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5978e54a-8766-4def-876e-91284b627cda None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:42.241886 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5978e54a-8766-4def-876e-91284b627cda None admin] PATH_INFO: `/v3/projects/b2bd4ad0f0dd4069ba3336740f72da62` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:42.243047 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5978e54a-8766-4def-876e-91284b627cda None admin] RBAC: Authorizing `identity:get_project(project_id=b2bd4ad0f0dd4069ba3336740f72da62)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:07:42.244510 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5978e54a-8766-4def-876e-91284b627cda None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:07:42.246140 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 602/1191] 10.222.0.61 () {56 vars in 1186 bytes} [Mon Aug 22 19:07:42 2022] GET /identity/v3/projects/b2bd4ad0f0dd4069ba3336740f72da62 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:07:42.251943 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c793afc-1fb7-4b2c-a0f6-175a39c7ee92 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:07:42.290207 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:07:42.348702 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c793afc-1fb7-4b2c-a0f6-175a39c7ee92 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:07:42.381077 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c164f00-0831-44ee-8267-e0263ef3f787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:07:42.381743 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c164f00-0831-44ee-8267-e0263ef3f787 None admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:42.382019 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c164f00-0831-44ee-8267-e0263ef3f787 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:42.382286 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c164f00-0831-44ee-8267-e0263ef3f787 None admin] PATH_INFO: `/v3/projects/b2bd4ad0f0dd4069ba3336740f72da62/users/17a182d20c5947858a5a5d53aeaadc5d/roles/163c30c3b3c246beb849ce176f02d9da` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:42.384189 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c164f00-0831-44ee-8267-e0263ef3f787 None admin] RBAC: Authorizing `identity:create_grant(project_id=b2bd4ad0f0dd4069ba3336740f72da62, user_id=17a182d20c5947858a5a5d53aeaadc5d, role_id=163c30c3b3c246beb849ce176f02d9da)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:07:42.385593 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c164f00-0831-44ee-8267-e0263ef3f787 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:07:42.397351 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 590/1192] 10.222.0.61 () {58 vars in 1439 bytes} [Mon Aug 22 19:07:42 2022] PUT /identity/v3/projects/b2bd4ad0f0dd4069ba3336740f72da62/users/17a182d20c5947858a5a5d53aeaadc5d/roles/163c30c3b3c246beb849ce176f02d9da => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:07:44.242419 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f373b7-453d-4b67-8620-c07cfeab3935 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:44.242941 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f373b7-453d-4b67-8620-c07cfeab3935 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:44.243043 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f373b7-453d-4b67-8620-c07cfeab3935 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:44.244367 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 603/1193] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:07:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:07:44.250196 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a9452d-b0b3-40f4-93ca-b1d76dd9f109 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:44.250585 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a9452d-b0b3-40f4-93ca-b1d76dd9f109 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:44.250744 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a9452d-b0b3-40f4-93ca-b1d76dd9f109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:44.268410 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-49a9452d-b0b3-40f4-93ca-b1d76dd9f109 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:07:44.269597 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-49a9452d-b0b3-40f4-93ca-b1d76dd9f109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:07:44.334165 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 591/1194] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:07:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 19:07:44.597805 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f373b7-453d-4b67-8620-c07cfeab3935 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:07:44.697332 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f373b7-453d-4b67-8620-c07cfeab3935 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:07:44.710305 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:07:44.730747 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02661155-d250-4f7f-a045-db913542eab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:07:44.731977 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02661155-d250-4f7f-a045-db913542eab6 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:44.732257 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02661155-d250-4f7f-a045-db913542eab6 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:44.732529 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02661155-d250-4f7f-a045-db913542eab6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:44.743107 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02661155-d250-4f7f-a045-db913542eab6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 19:07:44.747898 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-02661155-d250-4f7f-a045-db913542eab6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 19:07:44.747898 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:07:44.747898 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:07:44.747898 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:07:44.747898 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:07:44.747898 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:07:44.747898 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:07:44.747898 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:07:44.747898 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:07:44.747898 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:07:44.747898 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:07:44.747898 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:07:44.747898 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 19:07:44.747898 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 19:07:44.747898 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 19:07:44.747898 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 19:07:44.749019 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 19:07:44.749019 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 19:07:44.749019 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 19:07:44.749019 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 19:07:44.749019 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:07:44.749019 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:07:44.749019 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:07:44.749019 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:07:44.749019 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:07:44.749019 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:07:44.749019 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:07:44.749019 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:07:44.749019 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:07:44.749019 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:07:44.749019 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:07:44.749019 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:07:44.750067 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:07:44.750067 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:07:44.750067 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:07:44.750067 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:07:44.750067 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 19:07:44.750067 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 19:07:44.750067 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 19:07:44.750067 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 19:07:44.750067 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 19:07:44.750067 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 19:07:44.750067 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 19:07:44.750067 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:07:44.750067 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 604/1195] 10.222.0.61 () {56 vars in 1096 bytes} [Mon Aug 22 19:07:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:07:44.755154 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a9452d-b0b3-40f4-93ca-b1d76dd9f109 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:07:44.786613 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:07:44.857116 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a9452d-b0b3-40f4-93ca-b1d76dd9f109 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:07:44.893492 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab060e2-1416-459b-8ac1-a33b27f007a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:07:44.894096 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab060e2-1416-459b-8ac1-a33b27f007a2 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:44.894373 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab060e2-1416-459b-8ac1-a33b27f007a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:44.894643 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab060e2-1416-459b-8ac1-a33b27f007a2 None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:44.895307 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab060e2-1416-459b-8ac1-a33b27f007a2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:07:44.895605 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab060e2-1416-459b-8ac1-a33b27f007a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:07:44.896982 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab060e2-1416-459b-8ac1-a33b27f007a2 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:07:44.905400 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 592/1196] 10.222.0.61 () {56 vars in 1099 bytes} [Mon Aug 22 19:07:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:07:44.911355 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02661155-d250-4f7f-a045-db913542eab6 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:07:45.007180 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02661155-d250-4f7f-a045-db913542eab6 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:07:45.039882 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81001e5-1523-4f71-96a6-84ce76fc6ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:07:45.040496 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81001e5-1523-4f71-96a6-84ce76fc6ea4 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:45.040773 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81001e5-1523-4f71-96a6-84ce76fc6ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:45.041046 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81001e5-1523-4f71-96a6-84ce76fc6ea4 None admin] PATH_INFO: `/v3/users/17a182d20c5947858a5a5d53aeaadc5d` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:45.042161 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81001e5-1523-4f71-96a6-84ce76fc6ea4 None admin] RBAC: Authorizing `identity:get_user(user_id=17a182d20c5947858a5a5d53aeaadc5d)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:07:45.043605 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81001e5-1523-4f71-96a6-84ce76fc6ea4 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:07:45.045192 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 605/1197] 10.222.0.61 () {56 vars in 1177 bytes} [Mon Aug 22 19:07:44 2022] GET /identity/v3/users/17a182d20c5947858a5a5d53aeaadc5d => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:07:45.050875 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab060e2-1416-459b-8ac1-a33b27f007a2 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:07:45.137899 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:07:45.148464 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab060e2-1416-459b-8ac1-a33b27f007a2 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:07:45.181080 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ffc0a0-6300-4809-a805-6ec558c067eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:07:45.181676 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ffc0a0-6300-4809-a805-6ec558c067eb None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:45.181956 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ffc0a0-6300-4809-a805-6ec558c067eb None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:45.182225 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ffc0a0-6300-4809-a805-6ec558c067eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:45.247257 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ffc0a0-6300-4809-a805-6ec558c067eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:07:45.248706 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ffc0a0-6300-4809-a805-6ec558c067eb None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:07:45.333228 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:07:45.343934 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 593/1198] 10.222.0.61 () {58 vars in 1282 bytes} [Mon Aug 22 19:07:45 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 22 19:07:45.349627 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81001e5-1523-4f71-96a6-84ce76fc6ea4 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:07:45.392195 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:07:45.446555 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81001e5-1523-4f71-96a6-84ce76fc6ea4 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:07:45.478822 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d17880-05fd-4b75-acf1-d26902a8addd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:07:45.479439 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d17880-05fd-4b75-acf1-d26902a8addd None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:45.479739 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d17880-05fd-4b75-acf1-d26902a8addd None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:45.480012 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d17880-05fd-4b75-acf1-d26902a8addd None admin] PATH_INFO: `/v3/projects/b2bd4ad0f0dd4069ba3336740f72da62` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:45.481127 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d17880-05fd-4b75-acf1-d26902a8addd None admin] RBAC: Authorizing `identity:get_project(project_id=b2bd4ad0f0dd4069ba3336740f72da62)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:07:45.482540 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d17880-05fd-4b75-acf1-d26902a8addd None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:07:45.484159 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 606/1199] 10.222.0.61 () {56 vars in 1186 bytes} [Mon Aug 22 19:07:45 2022] GET /identity/v3/projects/b2bd4ad0f0dd4069ba3336740f72da62 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:07:45.490330 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ffc0a0-6300-4809-a805-6ec558c067eb None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:07:45.586424 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ffc0a0-6300-4809-a805-6ec558c067eb None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:07:45.620947 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e49bccb-e14e-4129-bc9f-9c7f0b113b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:07:45.622088 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e49bccb-e14e-4129-bc9f-9c7f0b113b5d None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:07:45.622366 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e49bccb-e14e-4129-bc9f-9c7f0b113b5d None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:07:45.622636 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e49bccb-e14e-4129-bc9f-9c7f0b113b5d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:07:45.623357 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e49bccb-e14e-4129-bc9f-9c7f0b113b5d None admin] RBAC: Adding query filter params (role.id=163c30c3b3c246beb849ce176f02d9da, scope.project.id=b2bd4ad0f0dd4069ba3336740f72da62, user.id=17a182d20c5947858a5a5d53aeaadc5d) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:07:45.623684 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e49bccb-e14e-4129-bc9f-9c7f0b113b5d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:07:45.625108 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e49bccb-e14e-4129-bc9f-9c7f0b113b5d None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:07:45.630755 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 594/1200] 10.222.0.61 () {56 vars in 1374 bytes} [Mon Aug 22 19:07:45 2022] GET /identity/v3/role_assignments?user.id=17a182d20c5947858a5a5d53aeaadc5d&scope.project.id=b2bd4ad0f0dd4069ba3336740f72da62&role.id=163c30c3b3c246beb849ce176f02d9da => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:07:55.474772 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=45 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:07:55.628236 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=73 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:08:40.230976 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b004e7-6eb9-4dc3-9ea0-8a0d5143a31d None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:08:40.230976 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b004e7-6eb9-4dc3-9ea0-8a0d5143a31d None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:08:40.230976 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b004e7-6eb9-4dc3-9ea0-8a0d5143a31d None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:08:40.230976 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 607/1201] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:08:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:08:40.236721 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abb71a4-73b5-4a94-a399-95ea4c03802d None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:08:40.237212 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abb71a4-73b5-4a94-a399-95ea4c03802d None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:08:40.237492 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abb71a4-73b5-4a94-a399-95ea4c03802d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:08:40.258275 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-8abb71a4-73b5-4a94-a399-95ea4c03802d None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:08:40.259793 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-8abb71a4-73b5-4a94-a399-95ea4c03802d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:08:40.347017 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 595/1202] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:08:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 19:08:40.614055 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b004e7-6eb9-4dc3-9ea0-8a0d5143a31d None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:08:40.747519 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b004e7-6eb9-4dc3-9ea0-8a0d5143a31d None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:08:40.794443 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af94a1a2-8d68-445c-a403-8e549528c0c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:08:40.795128 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af94a1a2-8d68-445c-a403-8e549528c0c1 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:08:40.795409 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af94a1a2-8d68-445c-a403-8e549528c0c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:08:40.795839 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af94a1a2-8d68-445c-a403-8e549528c0c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:08:40.812127 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af94a1a2-8d68-445c-a403-8e549528c0c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:08:40.814194 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af94a1a2-8d68-445c-a403-8e549528c0c1 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:08:40.835377 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-af94a1a2-8d68-445c-a403-8e549528c0c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:08:40.835377 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:08:40.835377 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:08:40.835377 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:08:40.835377 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:08:40.835377 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:08:40.835377 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:08:40.835377 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:08:40.835377 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:08:40.835377 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:08:40.835377 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:08:40.835377 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:08:40.835377 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 19:08:40.835377 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 19:08:40.835377 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 19:08:40.835377 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 19:08:40.836580 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:08:40.836580 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:08:40.836580 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:08:40.836580 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:08:40.836580 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:08:40.836580 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:08:40.836580 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:08:40.836580 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:08:40.836580 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:08:40.836580 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:08:40.836580 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:08:40.836580 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:08:40.836580 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:08:40.836580 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:08:40.836580 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:08:40.836580 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:08:40.836580 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 19:08:40.840895 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 19:08:40.840895 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:08:40.840895 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:08:40.840895 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 608/1203] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:08:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:08:40.843383 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abb71a4-73b5-4a94-a399-95ea4c03802d None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:08:40.975307 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abb71a4-73b5-4a94-a399-95ea4c03802d None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:08:41.011565 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:08:41.018280 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19530b59-d9d6-4aeb-922a-a0b970851bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:08:41.018951 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19530b59-d9d6-4aeb-922a-a0b970851bf7 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:08:41.019272 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19530b59-d9d6-4aeb-922a-a0b970851bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:08:41.019863 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19530b59-d9d6-4aeb-922a-a0b970851bf7 None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:08:41.020571 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19530b59-d9d6-4aeb-922a-a0b970851bf7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:08:41.020855 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19530b59-d9d6-4aeb-922a-a0b970851bf7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:08:41.022246 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19530b59-d9d6-4aeb-922a-a0b970851bf7 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:08:41.034484 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 596/1204] 10.222.0.61 () {56 vars in 1109 bytes} [Mon Aug 22 19:08:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:08:41.040921 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af94a1a2-8d68-445c-a403-8e549528c0c1 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:08:41.113493 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:08:41.179716 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af94a1a2-8d68-445c-a403-8e549528c0c1 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:08:41.222676 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cc0b35-ce46-4b63-baf5-590784b5dd79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:08:41.223794 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cc0b35-ce46-4b63-baf5-590784b5dd79 None admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:08:41.224402 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cc0b35-ce46-4b63-baf5-590784b5dd79 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:08:41.224912 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cc0b35-ce46-4b63-baf5-590784b5dd79 None admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:08:41.225927 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cc0b35-ce46-4b63-baf5-590784b5dd79 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:08:41.227386 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cc0b35-ce46-4b63-baf5-590784b5dd79 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:08:41.253703 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 609/1205] 10.222.0.61 () {60 vars in 1132 bytes} [Mon Aug 22 19:08:41 2022] POST /identity/v3/users => generated 251 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 19:08:43.314111 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55df77a-60f2-4fea-8adc-7dce5b69b27d None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:08:43.314863 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55df77a-60f2-4fea-8adc-7dce5b69b27d None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:08:43.314863 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55df77a-60f2-4fea-8adc-7dce5b69b27d None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:08:43.316045 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 597/1206] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:08:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:08:43.321373 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bf6c49-5ff4-419a-90db-862671b8d3db None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:08:43.321852 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bf6c49-5ff4-419a-90db-862671b8d3db None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:08:43.321953 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bf6c49-5ff4-419a-90db-862671b8d3db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:08:43.342007 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-09bf6c49-5ff4-419a-90db-862671b8d3db None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:08:43.343404 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-09bf6c49-5ff4-419a-90db-862671b8d3db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:08:43.423761 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 610/1207] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:08:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 19:08:43.700354 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55df77a-60f2-4fea-8adc-7dce5b69b27d None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:08:43.825113 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55df77a-60f2-4fea-8adc-7dce5b69b27d None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:08:43.864337 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ed007e-3c65-47bc-a374-0460af533911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:08:43.865117 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ed007e-3c65-47bc-a374-0460af533911 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:08:43.865462 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ed007e-3c65-47bc-a374-0460af533911 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:08:43.865808 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ed007e-3c65-47bc-a374-0460af533911 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:08:43.878153 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ed007e-3c65-47bc-a374-0460af533911 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 19:08:43.883480 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-18ed007e-3c65-47bc-a374-0460af533911 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 19:08:43.883480 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:08:43.883480 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:08:43.883480 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:08:43.883480 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:08:43.883480 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:08:43.883480 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:08:43.883480 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:08:43.883480 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:08:43.883480 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:08:43.883480 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:08:43.883480 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:08:43.883480 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 19:08:43.883480 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 19:08:43.883480 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 19:08:43.883480 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 19:08:43.884653 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 19:08:43.884653 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 19:08:43.884653 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 19:08:43.884653 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 19:08:43.884653 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:08:43.884653 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:08:43.884653 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:08:43.884653 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:08:43.884653 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:08:43.884653 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:08:43.884653 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:08:43.884653 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:08:43.884653 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:08:43.884653 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:08:43.884653 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:08:43.884653 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:08:43.886268 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:08:43.886268 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:08:43.886268 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:08:43.886268 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:08:43.886268 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 19:08:43.886268 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 19:08:43.886268 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 19:08:43.886268 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 19:08:43.886268 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 19:08:43.886268 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 19:08:43.886268 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 19:08:43.886268 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:08:43.886268 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 598/1208] 10.222.0.61 () {56 vars in 1102 bytes} [Mon Aug 22 19:08:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:08:43.892481 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bf6c49-5ff4-419a-90db-862671b8d3db None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:08:43.942075 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:08:44.019255 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bf6c49-5ff4-419a-90db-862671b8d3db None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:08:44.058764 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b567a2b-4516-49cc-8114-8417078d1fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:08:44.059394 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b567a2b-4516-49cc-8114-8417078d1fda None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:08:44.059770 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b567a2b-4516-49cc-8114-8417078d1fda None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:08:44.059965 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b567a2b-4516-49cc-8114-8417078d1fda None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:08:44.060634 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b567a2b-4516-49cc-8114-8417078d1fda None admin] RBAC: Adding query filter params (name=service) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:08:44.060931 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b567a2b-4516-49cc-8114-8417078d1fda None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:08:44.062295 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b567a2b-4516-49cc-8114-8417078d1fda None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:08:44.070834 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 611/1209] 10.222.0.61 () {56 vars in 1103 bytes} [Mon Aug 22 19:08:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:08:44.077151 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ed007e-3c65-47bc-a374-0460af533911 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:08:44.142280 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:08:44.202730 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ed007e-3c65-47bc-a374-0460af533911 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:08:44.239443 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ffa99e-5b49-4bf9-8719-af466d0285a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:08:44.240195 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ffa99e-5b49-4bf9-8719-af466d0285a8 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:08:44.240469 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ffa99e-5b49-4bf9-8719-af466d0285a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:08:44.240735 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ffa99e-5b49-4bf9-8719-af466d0285a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:08:44.252028 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ffa99e-5b49-4bf9-8719-af466d0285a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:08:44.253466 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ffa99e-5b49-4bf9-8719-af466d0285a8 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:08:44.274263 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-66ffa99e-5b49-4bf9-8719-af466d0285a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:08:44.274263 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:08:44.274263 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:08:44.274263 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:08:44.274263 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:08:44.274263 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:08:44.274263 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:08:44.274263 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:08:44.274263 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:08:44.274263 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:08:44.274263 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:08:44.274263 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:08:44.274263 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 19:08:44.274263 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 19:08:44.274263 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 19:08:44.274263 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 19:08:44.275359 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:08:44.275359 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:08:44.275359 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:08:44.275359 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:08:44.275359 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:08:44.275359 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:08:44.275359 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:08:44.275359 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:08:44.275359 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:08:44.275359 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:08:44.275359 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:08:44.275359 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:08:44.275359 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:08:44.275359 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:08:44.275359 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:08:44.275359 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:08:44.275359 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 19:08:44.282415 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 19:08:44.282415 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:08:44.282415 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:08:44.282415 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 599/1210] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:08:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 19:08:44.282736 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b567a2b-4516-49cc-8114-8417078d1fda None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:08:44.387115 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:08:44.391595 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b567a2b-4516-49cc-8114-8417078d1fda None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:08:44.430411 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024b0281-7945-4c46-8b1a-2d183d74174b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:08:44.431333 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024b0281-7945-4c46-8b1a-2d183d74174b None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:08:44.431620 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024b0281-7945-4c46-8b1a-2d183d74174b None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:08:44.431907 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024b0281-7945-4c46-8b1a-2d183d74174b None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:08:44.432550 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024b0281-7945-4c46-8b1a-2d183d74174b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:08:44.432851 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024b0281-7945-4c46-8b1a-2d183d74174b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:08:44.434204 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024b0281-7945-4c46-8b1a-2d183d74174b None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:08:44.451212 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 612/1211] 10.222.0.61 () {56 vars in 1109 bytes} [Mon Aug 22 19:08:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:08:44.452695 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ffa99e-5b49-4bf9-8719-af466d0285a8 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:08:44.561116 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ffa99e-5b49-4bf9-8719-af466d0285a8 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:08:44.562677 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:08:44.598228 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f11b8bc-71df-47f9-9b44-e3b0936662e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:08:44.599039 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f11b8bc-71df-47f9-9b44-e3b0936662e4 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:08:44.599499 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f11b8bc-71df-47f9-9b44-e3b0936662e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:08:44.599785 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f11b8bc-71df-47f9-9b44-e3b0936662e4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:08:44.607965 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f11b8bc-71df-47f9-9b44-e3b0936662e4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:08:44.609459 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f11b8bc-71df-47f9-9b44-e3b0936662e4 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:08:44.623037 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-7f11b8bc-71df-47f9-9b44-e3b0936662e4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 22 19:08:44.623037 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:08:44.623037 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:08:44.623037 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:08:44.623037 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:08:44.623037 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:08:44.623037 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:08:44.623037 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:08:44.623037 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:08:44.623037 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:08:44.623037 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:08:44.623037 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:08:44.623037 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 19:08:44.623037 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 19:08:44.623037 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 19:08:44.623037 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 19:08:44.626475 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:08:44.626475 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:08:44.626475 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 19:08:44.626475 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 19:08:44.626475 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 19:08:44.626475 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 19:08:44.626475 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:08:44.626475 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:08:44.626475 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:08:44.626475 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:08:44.626475 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:08:44.626475 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:08:44.626475 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:08:44.626475 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:08:44.626475 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:08:44.626475 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:08:44.626475 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:08:44.626475 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:08:44.627687 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:08:44.627687 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:08:44.627687 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 19:08:44.627687 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 19:08:44.627687 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 19:08:44.627687 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 19:08:44.627687 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 19:08:44.627687 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 19:08:44.627687 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 22 19:08:44.627687 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:08:44.627687 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 600/1212] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:08:44 2022] GET /identity/v3/users/placement => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:08:44.630003 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024b0281-7945-4c46-8b1a-2d183d74174b None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:08:44.742441 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024b0281-7945-4c46-8b1a-2d183d74174b None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:08:44.794509 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe82195c-49ec-492e-aa06-1bcaf71932d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:08:44.795156 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe82195c-49ec-492e-aa06-1bcaf71932d2 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:08:44.795437 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe82195c-49ec-492e-aa06-1bcaf71932d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:08:44.795742 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe82195c-49ec-492e-aa06-1bcaf71932d2 None admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:08:44.796615 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe82195c-49ec-492e-aa06-1bcaf71932d2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:08:44.796965 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe82195c-49ec-492e-aa06-1bcaf71932d2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:08:44.798509 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe82195c-49ec-492e-aa06-1bcaf71932d2 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:08:44.817996 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 613/1213] 10.222.0.61 () {56 vars in 1143 bytes} [Mon Aug 22 19:08:44 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:08:44.824664 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f11b8bc-71df-47f9-9b44-e3b0936662e4 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:08:44.936558 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f11b8bc-71df-47f9-9b44-e3b0936662e4 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:08:44.976591 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb87380-e5b9-4cf1-93f8-5e01fd9d7458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:08:44.977237 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb87380-e5b9-4cf1-93f8-5e01fd9d7458 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:08:44.978745 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb87380-e5b9-4cf1-93f8-5e01fd9d7458 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:08:44.978745 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb87380-e5b9-4cf1-93f8-5e01fd9d7458 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:08:45.023810 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:08:45.078415 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb87380-e5b9-4cf1-93f8-5e01fd9d7458 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:08:45.079870 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb87380-e5b9-4cf1-93f8-5e01fd9d7458 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:08:45.199241 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 601/1214] 10.222.0.61 () {58 vars in 1282 bytes} [Mon Aug 22 19:08:44 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 22 19:08:45.206257 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe82195c-49ec-492e-aa06-1bcaf71932d2 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:08:45.233517 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:08:45.331366 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe82195c-49ec-492e-aa06-1bcaf71932d2 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:08:45.376439 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e6f598-f655-4d82-858a-dd710f7bb963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:08:45.377084 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e6f598-f655-4d82-858a-dd710f7bb963 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:08:45.377503 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e6f598-f655-4d82-858a-dd710f7bb963 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:08:45.377834 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e6f598-f655-4d82-858a-dd710f7bb963 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:08:45.390920 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e6f598-f655-4d82-858a-dd710f7bb963 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:08:45.392666 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e6f598-f655-4d82-858a-dd710f7bb963 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:08:45.408944 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-c1e6f598-f655-4d82-858a-dd710f7bb963 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:08:45.408944 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:08:45.408944 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:08:45.408944 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:08:45.408944 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:08:45.408944 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:08:45.408944 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:08:45.408944 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:08:45.408944 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:08:45.408944 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:08:45.408944 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:08:45.408944 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:08:45.408944 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 19:08:45.408944 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 19:08:45.408944 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 19:08:45.408944 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 19:08:45.410388 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:08:45.410388 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:08:45.410388 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:08:45.410388 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:08:45.410388 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:08:45.410388 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:08:45.410388 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:08:45.410388 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:08:45.410388 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:08:45.410388 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:08:45.410388 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:08:45.410388 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:08:45.410388 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:08:45.410388 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:08:45.410388 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:08:45.410388 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:08:45.410388 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 19:08:45.411533 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 19:08:45.411533 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:08:45.411533 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:08:45.411533 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 614/1215] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:08:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:08:45.417239 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb87380-e5b9-4cf1-93f8-5e01fd9d7458 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:08:45.490323 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:08:45.543686 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb87380-e5b9-4cf1-93f8-5e01fd9d7458 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:08:45.586800 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f0b88d-7d96-473e-ba5b-e68bd417f7dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:08:45.587281 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f0b88d-7d96-473e-ba5b-e68bd417f7dd None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:08:45.587539 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f0b88d-7d96-473e-ba5b-e68bd417f7dd None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:08:45.588466 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f0b88d-7d96-473e-ba5b-e68bd417f7dd None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:08:45.589124 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f0b88d-7d96-473e-ba5b-e68bd417f7dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:08:45.589414 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f0b88d-7d96-473e-ba5b-e68bd417f7dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:08:45.590764 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f0b88d-7d96-473e-ba5b-e68bd417f7dd None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:08:45.607301 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 602/1216] 10.222.0.61 () {56 vars in 1109 bytes} [Mon Aug 22 19:08:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:08:45.608647 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e6f598-f655-4d82-858a-dd710f7bb963 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:08:45.688366 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:08:45.725321 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e6f598-f655-4d82-858a-dd710f7bb963 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:08:45.761788 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a1d620-2170-476d-9814-914dd863f203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:08:45.762505 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a1d620-2170-476d-9814-914dd863f203 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:08:45.762835 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a1d620-2170-476d-9814-914dd863f203 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:08:45.763172 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a1d620-2170-476d-9814-914dd863f203 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:08:45.770092 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a1d620-2170-476d-9814-914dd863f203 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:08:45.771590 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a1d620-2170-476d-9814-914dd863f203 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:08:45.783365 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-41a1d620-2170-476d-9814-914dd863f203 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 19:08:45.783365 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:08:45.783365 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:08:45.783365 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:08:45.783365 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:08:45.783365 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:08:45.783365 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:08:45.783365 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:08:45.783365 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:08:45.783365 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:08:45.783365 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:08:45.783365 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:08:45.783365 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 19:08:45.783365 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 19:08:45.783365 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 19:08:45.784563 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 19:08:45.784563 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:08:45.784563 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:08:45.784563 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:08:45.784563 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:08:45.784563 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:08:45.784563 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:08:45.784563 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:08:45.784563 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:08:45.784563 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:08:45.784563 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:08:45.784563 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:08:45.784563 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:08:45.784563 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:08:45.784563 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:08:45.784563 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:08:45.784563 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:08:45.787803 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 19:08:45.787803 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 19:08:45.787803 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 19:08:45.787803 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 19:08:45.787803 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 19:08:45.787803 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 19:08:45.787803 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 19:08:45.787803 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:08:45.787803 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 615/1217] 10.222.0.61 () {56 vars in 1111 bytes} [Mon Aug 22 19:08:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:08:45.793510 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f0b88d-7d96-473e-ba5b-e68bd417f7dd None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:08:45.908977 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f0b88d-7d96-473e-ba5b-e68bd417f7dd None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:08:45.933229 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:08:45.946897 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f92d03-8267-4739-94ff-321d5af3f83e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:08:45.948051 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f92d03-8267-4739-94ff-321d5af3f83e None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:08:45.948326 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f92d03-8267-4739-94ff-321d5af3f83e None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:08:45.948591 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f92d03-8267-4739-94ff-321d5af3f83e None admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:08:45.949273 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f92d03-8267-4739-94ff-321d5af3f83e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:08:45.949571 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f92d03-8267-4739-94ff-321d5af3f83e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:08:45.950977 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f92d03-8267-4739-94ff-321d5af3f83e None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:08:45.969188 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 603/1218] 10.222.0.61 () {56 vars in 1148 bytes} [Mon Aug 22 19:08:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:08:45.970831 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a1d620-2170-476d-9814-914dd863f203 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:08:46.096360 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a1d620-2170-476d-9814-914dd863f203 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:08:46.135598 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:08:46.147393 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027e8d6e-7ca0-4009-8b2f-72b9d1855ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:08:46.148116 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027e8d6e-7ca0-4009-8b2f-72b9d1855ea7 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:08:46.148463 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027e8d6e-7ca0-4009-8b2f-72b9d1855ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:08:46.148806 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027e8d6e-7ca0-4009-8b2f-72b9d1855ea7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:08:46.149619 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027e8d6e-7ca0-4009-8b2f-72b9d1855ea7 None admin] RBAC: Adding query filter params (role.id=324bd04d55c34b61a5282c9f6ed73f46, scope.project.id=e07dc5ebdc504c6da706a2d21c0a3970, user.id=eef0f73297e04d62bc22acb790fad6ab) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:08:46.150000 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027e8d6e-7ca0-4009-8b2f-72b9d1855ea7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:08:46.152184 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027e8d6e-7ca0-4009-8b2f-72b9d1855ea7 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:08:46.167758 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 616/1219] 10.222.0.61 () {56 vars in 1374 bytes} [Mon Aug 22 19:08:45 2022] GET /identity/v3/role_assignments?user.id=eef0f73297e04d62bc22acb790fad6ab&scope.project.id=e07dc5ebdc504c6da706a2d21c0a3970&role.id=324bd04d55c34b61a5282c9f6ed73f46 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 19:08:48.035743 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb53ce79-dbaf-4365-8fc4-0a38457da39c None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:08:48.036333 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb53ce79-dbaf-4365-8fc4-0a38457da39c None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:08:48.036333 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb53ce79-dbaf-4365-8fc4-0a38457da39c None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:08:48.037609 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 604/1220] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:08:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 19:08:48.043319 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c990a7f-ef45-40e7-ad1d-2e11ef3a6340 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:08:48.043823 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c990a7f-ef45-40e7-ad1d-2e11ef3a6340 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:08:48.043955 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c990a7f-ef45-40e7-ad1d-2e11ef3a6340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:08:48.062622 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-4c990a7f-ef45-40e7-ad1d-2e11ef3a6340 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:08:48.063828 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-4c990a7f-ef45-40e7-ad1d-2e11ef3a6340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:08:48.137095 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 617/1221] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:08:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 19:08:48.400034 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb53ce79-dbaf-4365-8fc4-0a38457da39c None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:08:48.500632 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb53ce79-dbaf-4365-8fc4-0a38457da39c None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:08:48.535876 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7ffdc2-30e1-48bf-a313-39d31694855f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:08:48.536507 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7ffdc2-30e1-48bf-a313-39d31694855f None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:08:48.536786 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7ffdc2-30e1-48bf-a313-39d31694855f None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:08:48.537043 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7ffdc2-30e1-48bf-a313-39d31694855f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:08:48.548647 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7ffdc2-30e1-48bf-a313-39d31694855f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 19:08:48.553476 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-5a7ffdc2-30e1-48bf-a313-39d31694855f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 19:08:48.553476 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:08:48.553476 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:08:48.553476 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:08:48.553476 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:08:48.553476 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:08:48.553476 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:08:48.553476 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:08:48.553476 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:08:48.553476 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:08:48.553476 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:08:48.553476 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:08:48.553476 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 19:08:48.553476 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 19:08:48.553476 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 19:08:48.553476 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 19:08:48.554702 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 19:08:48.554702 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 19:08:48.554702 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 19:08:48.554702 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 19:08:48.554702 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:08:48.554702 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:08:48.554702 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:08:48.554702 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:08:48.554702 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:08:48.554702 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:08:48.554702 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:08:48.554702 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:08:48.554702 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:08:48.554702 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:08:48.554702 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:08:48.554702 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:08:48.555804 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:08:48.555804 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:08:48.555804 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:08:48.555804 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:08:48.555804 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 19:08:48.555804 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 19:08:48.555804 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 19:08:48.555804 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 19:08:48.555804 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 19:08:48.555804 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 19:08:48.555804 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 19:08:48.555804 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:08:48.560344 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 605/1222] 10.222.0.61 () {56 vars in 1102 bytes} [Mon Aug 22 19:08:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:08:48.561472 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c990a7f-ef45-40e7-ad1d-2e11ef3a6340 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:08:48.665415 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c990a7f-ef45-40e7-ad1d-2e11ef3a6340 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:08:48.691374 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:08:48.702338 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae586907-dda1-4f74-a70d-63fda12ff2c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:08:48.702941 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae586907-dda1-4f74-a70d-63fda12ff2c3 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:08:48.703216 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae586907-dda1-4f74-a70d-63fda12ff2c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:08:48.703483 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae586907-dda1-4f74-a70d-63fda12ff2c3 None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:08:48.704175 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae586907-dda1-4f74-a70d-63fda12ff2c3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:08:48.704474 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae586907-dda1-4f74-a70d-63fda12ff2c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:08:48.705828 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae586907-dda1-4f74-a70d-63fda12ff2c3 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:08:48.713692 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 618/1223] 10.222.0.61 () {56 vars in 1103 bytes} [Mon Aug 22 19:08:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:08:48.719330 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7ffdc2-30e1-48bf-a313-39d31694855f None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:08:48.767462 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:08:48.827278 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7ffdc2-30e1-48bf-a313-39d31694855f None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:08:48.862248 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ab6345-0001-4a3a-bdd3-e42b74c9c511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:08:48.862881 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ab6345-0001-4a3a-bdd3-e42b74c9c511 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:08:48.863163 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ab6345-0001-4a3a-bdd3-e42b74c9c511 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:08:48.863436 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ab6345-0001-4a3a-bdd3-e42b74c9c511 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:08:48.874334 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ab6345-0001-4a3a-bdd3-e42b74c9c511 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:08:48.875855 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ab6345-0001-4a3a-bdd3-e42b74c9c511 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:08:48.890211 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-56ab6345-0001-4a3a-bdd3-e42b74c9c511 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:08:48.890211 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:08:48.890211 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:08:48.890211 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:08:48.890211 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:08:48.890211 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:08:48.890211 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:08:48.890211 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:08:48.890211 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:08:48.890211 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:08:48.890211 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:08:48.890211 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:08:48.890211 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 19:08:48.890211 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 19:08:48.890211 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 19:08:48.890211 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 19:08:48.891625 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:08:48.891625 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:08:48.891625 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:08:48.891625 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:08:48.891625 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:08:48.891625 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:08:48.891625 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:08:48.891625 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:08:48.891625 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:08:48.891625 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:08:48.891625 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:08:48.891625 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:08:48.891625 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:08:48.891625 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:08:48.891625 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:08:48.891625 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:08:48.891625 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 19:08:48.892860 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 19:08:48.892860 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:08:48.892860 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:08:48.892860 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 606/1224] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:08:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:08:48.897279 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae586907-dda1-4f74-a70d-63fda12ff2c3 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:08:49.000928 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae586907-dda1-4f74-a70d-63fda12ff2c3 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:08:49.036555 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe4f221-05f3-4d36-a164-f7e6099082cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:08:49.037168 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe4f221-05f3-4d36-a164-f7e6099082cf None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:08:49.037447 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe4f221-05f3-4d36-a164-f7e6099082cf None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:08:49.037720 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe4f221-05f3-4d36-a164-f7e6099082cf None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:08:49.038367 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe4f221-05f3-4d36-a164-f7e6099082cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:08:49.038670 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe4f221-05f3-4d36-a164-f7e6099082cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:08:49.040075 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe4f221-05f3-4d36-a164-f7e6099082cf None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:08:49.052148 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 619/1225] 10.222.0.61 () {56 vars in 1109 bytes} [Mon Aug 22 19:08:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:08:49.058514 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ab6345-0001-4a3a-bdd3-e42b74c9c511 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:08:49.165031 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:08:49.172149 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ab6345-0001-4a3a-bdd3-e42b74c9c511 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:08:49.212884 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefa44a2-1bff-46fb-bfd0-0f548474593a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:08:49.213751 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefa44a2-1bff-46fb-bfd0-0f548474593a None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:08:49.214072 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefa44a2-1bff-46fb-bfd0-0f548474593a None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:08:49.214401 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefa44a2-1bff-46fb-bfd0-0f548474593a None admin] PATH_INFO: `/v3/users/placement` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:08:49.222978 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefa44a2-1bff-46fb-bfd0-0f548474593a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:08:49.224614 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefa44a2-1bff-46fb-bfd0-0f548474593a None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:08:49.237443 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-eefa44a2-1bff-46fb-bfd0-0f548474593a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 22 19:08:49.237443 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:08:49.237443 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:08:49.237443 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:08:49.237443 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:08:49.237443 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:08:49.237443 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:08:49.237443 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:08:49.237443 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:08:49.237443 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:08:49.237443 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:08:49.237443 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:08:49.237443 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 19:08:49.237443 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 19:08:49.237443 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 19:08:49.237443 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 19:08:49.238558 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:08:49.238558 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:08:49.238558 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 19:08:49.238558 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 19:08:49.238558 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 19:08:49.238558 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 19:08:49.238558 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:08:49.238558 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:08:49.238558 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:08:49.238558 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:08:49.238558 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:08:49.238558 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:08:49.238558 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:08:49.238558 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:08:49.238558 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:08:49.238558 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:08:49.238558 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:08:49.238558 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:08:49.239624 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:08:49.239624 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:08:49.239624 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 19:08:49.239624 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 19:08:49.239624 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 19:08:49.239624 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 19:08:49.239624 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 19:08:49.239624 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 19:08:49.239624 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 22 19:08:49.239624 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:08:49.239624 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 607/1226] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:08:49 2022] GET /identity/v3/users/placement => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:08:49.245295 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe4f221-05f3-4d36-a164-f7e6099082cf None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:08:49.295252 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:08:49.358619 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe4f221-05f3-4d36-a164-f7e6099082cf None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:08:49.394427 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c977a376-0066-422e-ad06-3f7112d6e641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:08:49.395030 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c977a376-0066-422e-ad06-3f7112d6e641 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:08:49.395303 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c977a376-0066-422e-ad06-3f7112d6e641 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:08:49.395574 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c977a376-0066-422e-ad06-3f7112d6e641 None admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:08:49.396356 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c977a376-0066-422e-ad06-3f7112d6e641 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:08:49.396628 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c977a376-0066-422e-ad06-3f7112d6e641 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:08:49.398044 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c977a376-0066-422e-ad06-3f7112d6e641 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:08:49.416283 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 620/1227] 10.222.0.61 () {56 vars in 1143 bytes} [Mon Aug 22 19:08:49 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:08:49.422521 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefa44a2-1bff-46fb-bfd0-0f548474593a None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:08:49.538942 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefa44a2-1bff-46fb-bfd0-0f548474593a None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:08:49.576901 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1da97b7-5061-4797-801e-fb3b9d69455b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:08:49.577495 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1da97b7-5061-4797-801e-fb3b9d69455b None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:08:49.577776 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1da97b7-5061-4797-801e-fb3b9d69455b None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:08:49.578048 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1da97b7-5061-4797-801e-fb3b9d69455b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:08:49.589361 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1da97b7-5061-4797-801e-fb3b9d69455b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:08:49.590919 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1da97b7-5061-4797-801e-fb3b9d69455b None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:08:49.606316 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-c1da97b7-5061-4797-801e-fb3b9d69455b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:08:49.606316 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:08:49.606316 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:08:49.606316 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:08:49.606316 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:08:49.606316 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:08:49.606316 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:08:49.606316 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:08:49.606316 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:08:49.606316 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:08:49.606316 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:08:49.606316 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:08:49.606316 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 19:08:49.606316 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 19:08:49.606316 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 19:08:49.606316 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 19:08:49.607340 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:08:49.607340 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:08:49.607340 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:08:49.607340 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:08:49.607340 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:08:49.607340 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:08:49.607340 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:08:49.607340 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:08:49.607340 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:08:49.607340 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:08:49.607340 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:08:49.607340 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:08:49.607340 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:08:49.607340 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:08:49.607340 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:08:49.607340 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:08:49.607340 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 19:08:49.608814 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 19:08:49.608814 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:08:49.608814 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:08:49.608814 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 608/1228] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:08:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:08:49.614018 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c977a376-0066-422e-ad06-3f7112d6e641 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:08:49.730745 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c977a376-0066-422e-ad06-3f7112d6e641 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:08:49.733409 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:08:49.769608 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf5c08c-c740-4f7a-a9ea-e55341744410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:08:49.770213 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf5c08c-c740-4f7a-a9ea-e55341744410 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:08:49.770488 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf5c08c-c740-4f7a-a9ea-e55341744410 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:08:49.770760 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf5c08c-c740-4f7a-a9ea-e55341744410 None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:08:49.771395 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf5c08c-c740-4f7a-a9ea-e55341744410 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:08:49.771712 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf5c08c-c740-4f7a-a9ea-e55341744410 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:08:49.773073 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf5c08c-c740-4f7a-a9ea-e55341744410 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:08:49.785219 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 621/1229] 10.222.0.61 () {56 vars in 1109 bytes} [Mon Aug 22 19:08:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:08:49.792646 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1da97b7-5061-4797-801e-fb3b9d69455b None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:08:49.803238 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:08:49.899927 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1da97b7-5061-4797-801e-fb3b9d69455b None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:08:49.938709 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bc6c8c-b2c4-42a9-916c-48b51f0cc5a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:08:49.939340 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bc6c8c-b2c4-42a9-916c-48b51f0cc5a0 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:08:49.939662 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bc6c8c-b2c4-42a9-916c-48b51f0cc5a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:08:49.939934 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bc6c8c-b2c4-42a9-916c-48b51f0cc5a0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:08:49.946603 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bc6c8c-b2c4-42a9-916c-48b51f0cc5a0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:08:49.948709 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bc6c8c-b2c4-42a9-916c-48b51f0cc5a0 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:08:49.960019 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-93bc6c8c-b2c4-42a9-916c-48b51f0cc5a0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 19:08:49.960019 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:08:49.960019 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:08:49.960019 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:08:49.960019 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:08:49.960019 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:08:49.960019 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:08:49.960019 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:08:49.960019 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:08:49.960019 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:08:49.960019 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:08:49.960019 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:08:49.960019 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 19:08:49.960019 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 19:08:49.960019 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 19:08:49.961147 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 19:08:49.961147 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:08:49.961147 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:08:49.961147 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:08:49.961147 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:08:49.961147 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:08:49.961147 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:08:49.961147 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:08:49.961147 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:08:49.961147 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:08:49.961147 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:08:49.961147 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:08:49.961147 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:08:49.961147 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:08:49.961147 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:08:49.961147 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:08:49.961147 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:08:49.962192 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 19:08:49.962192 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 19:08:49.962192 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 19:08:49.962192 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 19:08:49.962192 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 19:08:49.962192 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 19:08:49.962192 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 19:08:49.962192 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:08:49.962192 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 609/1230] 10.222.0.61 () {56 vars in 1111 bytes} [Mon Aug 22 19:08:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:08:49.967171 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf5c08c-c740-4f7a-a9ea-e55341744410 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:08:50.085493 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf5c08c-c740-4f7a-a9ea-e55341744410 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:08:50.124895 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b167ce0a-2814-4c7f-a355-7a8e01eab8b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:08:50.125496 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b167ce0a-2814-4c7f-a355-7a8e01eab8b2 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:08:50.125766 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b167ce0a-2814-4c7f-a355-7a8e01eab8b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:08:50.126037 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b167ce0a-2814-4c7f-a355-7a8e01eab8b2 None admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:08:50.126732 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b167ce0a-2814-4c7f-a355-7a8e01eab8b2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:08:50.127078 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b167ce0a-2814-4c7f-a355-7a8e01eab8b2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:08:50.128503 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b167ce0a-2814-4c7f-a355-7a8e01eab8b2 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:08:50.144483 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 622/1231] 10.222.0.61 () {56 vars in 1148 bytes} [Mon Aug 22 19:08:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 19:08:50.149079 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bc6c8c-b2c4-42a9-916c-48b51f0cc5a0 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:08:50.222141 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:08:50.262103 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bc6c8c-b2c4-42a9-916c-48b51f0cc5a0 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:08:50.298210 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebf0632-acbc-4ea1-a027-ef2c7fe7f4da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:08:50.298851 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebf0632-acbc-4ea1-a027-ef2c7fe7f4da None admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:08:50.299131 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebf0632-acbc-4ea1-a027-ef2c7fe7f4da None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:08:50.299399 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebf0632-acbc-4ea1-a027-ef2c7fe7f4da None admin] PATH_INFO: `/v3/projects/e07dc5ebdc504c6da706a2d21c0a3970/users/eef0f73297e04d62bc22acb790fad6ab/roles/324bd04d55c34b61a5282c9f6ed73f46` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:08:50.315690 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebf0632-acbc-4ea1-a027-ef2c7fe7f4da None admin] RBAC: Authorizing `identity:create_grant(project_id=e07dc5ebdc504c6da706a2d21c0a3970, user_id=eef0f73297e04d62bc22acb790fad6ab, role_id=324bd04d55c34b61a5282c9f6ed73f46)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:08:50.317148 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebf0632-acbc-4ea1-a027-ef2c7fe7f4da None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:08:50.330925 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 610/1232] 10.222.0.61 () {58 vars in 1439 bytes} [Mon Aug 22 19:08:50 2022] PUT /identity/v3/projects/e07dc5ebdc504c6da706a2d21c0a3970/users/eef0f73297e04d62bc22acb790fad6ab/roles/324bd04d55c34b61a5282c9f6ed73f46 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:08:52.338857 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4e3461-e021-43c3-b181-100c591b8479 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:08:52.339423 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4e3461-e021-43c3-b181-100c591b8479 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:08:52.339423 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4e3461-e021-43c3-b181-100c591b8479 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:08:52.340754 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 623/1233] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:08:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:08:52.346034 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7c183f-9221-4289-b388-7464ca156e3d None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:08:52.346461 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7c183f-9221-4289-b388-7464ca156e3d None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:08:52.346595 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7c183f-9221-4289-b388-7464ca156e3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:08:52.364496 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-2f7c183f-9221-4289-b388-7464ca156e3d None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:08:52.365682 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-2f7c183f-9221-4289-b388-7464ca156e3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:08:52.433898 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 611/1234] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:08:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 19:08:52.700474 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4e3461-e021-43c3-b181-100c591b8479 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:08:52.729360 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:08:52.812428 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4e3461-e021-43c3-b181-100c591b8479 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:08:52.848744 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21cbc77-ade8-4286-812b-2300dc3a1c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:08:52.849380 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21cbc77-ade8-4286-812b-2300dc3a1c80 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:08:52.849657 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21cbc77-ade8-4286-812b-2300dc3a1c80 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:08:52.849927 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21cbc77-ade8-4286-812b-2300dc3a1c80 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:08:52.860962 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21cbc77-ade8-4286-812b-2300dc3a1c80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 19:08:52.865740 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-f21cbc77-ade8-4286-812b-2300dc3a1c80 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 19:08:52.865740 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:08:52.865740 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:08:52.865740 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:08:52.865740 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:08:52.865740 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:08:52.865740 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:08:52.865740 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:08:52.865740 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:08:52.865740 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:08:52.865740 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:08:52.865740 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:08:52.865740 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 19:08:52.865740 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 19:08:52.865740 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 19:08:52.865740 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 19:08:52.866843 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 19:08:52.866843 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 19:08:52.866843 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 19:08:52.866843 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 19:08:52.866843 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:08:52.866843 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:08:52.866843 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:08:52.866843 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:08:52.866843 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:08:52.866843 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:08:52.866843 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:08:52.866843 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:08:52.866843 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:08:52.866843 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:08:52.866843 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:08:52.866843 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:08:52.867744 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:08:52.867744 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:08:52.867744 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:08:52.867744 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:08:52.867744 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 19:08:52.867744 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 19:08:52.867744 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 19:08:52.867744 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 19:08:52.867744 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 19:08:52.867744 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 19:08:52.867744 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 19:08:52.867744 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:08:52.867744 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 624/1235] 10.222.0.61 () {56 vars in 1102 bytes} [Mon Aug 22 19:08:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:08:52.873069 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7c183f-9221-4289-b388-7464ca156e3d None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:08:52.919175 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:08:52.976834 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7c183f-9221-4289-b388-7464ca156e3d None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:08:53.013018 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adedb9b5-2133-48ad-8d8a-d4cdcf3e52de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:08:53.013626 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adedb9b5-2133-48ad-8d8a-d4cdcf3e52de None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:08:53.013901 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adedb9b5-2133-48ad-8d8a-d4cdcf3e52de None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:08:53.014169 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adedb9b5-2133-48ad-8d8a-d4cdcf3e52de None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:08:53.014832 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adedb9b5-2133-48ad-8d8a-d4cdcf3e52de None admin] RBAC: Adding query filter params (name=service) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:08:53.015129 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adedb9b5-2133-48ad-8d8a-d4cdcf3e52de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:08:53.016527 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adedb9b5-2133-48ad-8d8a-d4cdcf3e52de None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:08:53.025693 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 612/1236] 10.222.0.61 () {56 vars in 1103 bytes} [Mon Aug 22 19:08:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:08:53.031987 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21cbc77-ade8-4286-812b-2300dc3a1c80 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:08:53.127326 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:08:53.145322 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21cbc77-ade8-4286-812b-2300dc3a1c80 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:08:53.183501 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56c4f4a-9425-4c3d-950f-22a59230a667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:08:53.184346 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56c4f4a-9425-4c3d-950f-22a59230a667 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:08:53.184650 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56c4f4a-9425-4c3d-950f-22a59230a667 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:08:53.184933 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56c4f4a-9425-4c3d-950f-22a59230a667 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:08:53.196439 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56c4f4a-9425-4c3d-950f-22a59230a667 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:08:53.198100 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56c4f4a-9425-4c3d-950f-22a59230a667 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:08:53.214560 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-d56c4f4a-9425-4c3d-950f-22a59230a667 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:08:53.214560 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:08:53.214560 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:08:53.214560 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:08:53.214560 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:08:53.214560 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:08:53.214560 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:08:53.214560 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:08:53.214560 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:08:53.214560 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:08:53.214560 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:08:53.214560 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:08:53.214560 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 19:08:53.214560 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 19:08:53.214560 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 19:08:53.214560 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 19:08:53.215778 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:08:53.215778 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:08:53.215778 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:08:53.215778 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:08:53.215778 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:08:53.215778 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:08:53.215778 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:08:53.215778 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:08:53.215778 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:08:53.215778 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:08:53.215778 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:08:53.215778 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:08:53.215778 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:08:53.215778 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:08:53.215778 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:08:53.215778 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:08:53.215778 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 19:08:53.216910 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 19:08:53.216910 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:08:53.216910 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:08:53.216910 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 625/1237] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:08:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:08:53.221687 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adedb9b5-2133-48ad-8d8a-d4cdcf3e52de None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:08:53.334241 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adedb9b5-2133-48ad-8d8a-d4cdcf3e52de None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:08:53.346064 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:08:53.372971 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78583b2e-01b6-4967-9dcb-9400c8c36c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:08:53.373573 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78583b2e-01b6-4967-9dcb-9400c8c36c12 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:08:53.373848 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78583b2e-01b6-4967-9dcb-9400c8c36c12 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:08:53.374115 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78583b2e-01b6-4967-9dcb-9400c8c36c12 None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:08:53.374761 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78583b2e-01b6-4967-9dcb-9400c8c36c12 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:08:53.375047 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78583b2e-01b6-4967-9dcb-9400c8c36c12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:08:53.376541 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78583b2e-01b6-4967-9dcb-9400c8c36c12 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:08:53.388205 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 613/1238] 10.222.0.61 () {56 vars in 1109 bytes} [Mon Aug 22 19:08:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:08:53.394147 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56c4f4a-9425-4c3d-950f-22a59230a667 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:08:53.511198 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56c4f4a-9425-4c3d-950f-22a59230a667 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:08:53.554418 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf20055-b6de-478f-bffb-5d9bc7413129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:08:53.555069 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf20055-b6de-478f-bffb-5d9bc7413129 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:08:53.555368 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf20055-b6de-478f-bffb-5d9bc7413129 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:08:53.555615 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf20055-b6de-478f-bffb-5d9bc7413129 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:08:53.558901 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:08:53.564616 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf20055-b6de-478f-bffb-5d9bc7413129 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:08:53.566364 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf20055-b6de-478f-bffb-5d9bc7413129 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:08:53.578452 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-cdf20055-b6de-478f-bffb-5d9bc7413129 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 22 19:08:53.578452 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:08:53.578452 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:08:53.578452 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:08:53.578452 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:08:53.578452 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:08:53.578452 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:08:53.578452 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:08:53.578452 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:08:53.578452 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:08:53.578452 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:08:53.578452 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:08:53.578452 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 19:08:53.578452 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 19:08:53.578452 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 19:08:53.578452 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 19:08:53.579727 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:08:53.579727 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:08:53.579727 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 19:08:53.579727 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 19:08:53.579727 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 19:08:53.579727 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 19:08:53.579727 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:08:53.579727 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:08:53.579727 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:08:53.579727 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:08:53.579727 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:08:53.579727 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:08:53.579727 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:08:53.579727 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:08:53.579727 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:08:53.579727 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:08:53.579727 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:08:53.579727 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:08:53.580911 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:08:53.580911 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:08:53.580911 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 19:08:53.580911 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 19:08:53.580911 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 19:08:53.580911 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 19:08:53.580911 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 19:08:53.580911 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 19:08:53.580911 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 22 19:08:53.580911 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:08:53.580911 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 626/1239] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:08:53 2022] GET /identity/v3/users/placement => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:08:53.585619 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78583b2e-01b6-4967-9dcb-9400c8c36c12 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:08:53.693979 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78583b2e-01b6-4967-9dcb-9400c8c36c12 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:08:53.730421 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a2f2c8-c5c3-429f-8389-2d3fdd564e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:08:53.731120 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a2f2c8-c5c3-429f-8389-2d3fdd564e70 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:08:53.731476 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a2f2c8-c5c3-429f-8389-2d3fdd564e70 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:08:53.731846 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a2f2c8-c5c3-429f-8389-2d3fdd564e70 None admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:08:53.732571 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a2f2c8-c5c3-429f-8389-2d3fdd564e70 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:08:53.732874 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a2f2c8-c5c3-429f-8389-2d3fdd564e70 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:08:53.734314 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a2f2c8-c5c3-429f-8389-2d3fdd564e70 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:08:53.756426 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 614/1240] 10.222.0.61 () {56 vars in 1143 bytes} [Mon Aug 22 19:08:53 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:08:53.757739 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf20055-b6de-478f-bffb-5d9bc7413129 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:08:53.869039 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf20055-b6de-478f-bffb-5d9bc7413129 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:08:53.907562 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c0e009-3ded-4c05-b10a-db68ac83ee76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:08:53.908359 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c0e009-3ded-4c05-b10a-db68ac83ee76 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:08:53.908702 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c0e009-3ded-4c05-b10a-db68ac83ee76 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:08:53.909152 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c0e009-3ded-4c05-b10a-db68ac83ee76 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:08:53.977999 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:08:53.985206 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c0e009-3ded-4c05-b10a-db68ac83ee76 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:08:53.986921 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c0e009-3ded-4c05-b10a-db68ac83ee76 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:08:54.093615 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 627/1241] 10.222.0.61 () {58 vars in 1282 bytes} [Mon Aug 22 19:08:53 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 22 19:08:54.099727 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a2f2c8-c5c3-429f-8389-2d3fdd564e70 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:08:54.128460 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:08:54.211277 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a2f2c8-c5c3-429f-8389-2d3fdd564e70 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:08:54.250276 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fa9a88-c929-4419-9eee-6c0d490a243f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:08:54.251024 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fa9a88-c929-4419-9eee-6c0d490a243f None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:08:54.251320 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fa9a88-c929-4419-9eee-6c0d490a243f None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:08:54.251673 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fa9a88-c929-4419-9eee-6c0d490a243f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:08:54.262589 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fa9a88-c929-4419-9eee-6c0d490a243f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:08:54.264170 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fa9a88-c929-4419-9eee-6c0d490a243f None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:08:54.279827 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-72fa9a88-c929-4419-9eee-6c0d490a243f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:08:54.279827 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:08:54.279827 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:08:54.279827 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:08:54.279827 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:08:54.279827 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:08:54.279827 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:08:54.279827 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:08:54.279827 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:08:54.279827 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:08:54.279827 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:08:54.279827 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:08:54.279827 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 19:08:54.279827 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 19:08:54.279827 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 19:08:54.279827 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 19:08:54.280950 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:08:54.280950 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:08:54.280950 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:08:54.280950 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:08:54.280950 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:08:54.280950 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:08:54.280950 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:08:54.280950 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:08:54.280950 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:08:54.280950 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:08:54.280950 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:08:54.280950 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:08:54.280950 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:08:54.280950 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:08:54.280950 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:08:54.280950 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:08:54.280950 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 19:08:54.282067 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 19:08:54.282067 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:08:54.282067 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:08:54.282067 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 615/1242] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:08:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:08:54.286593 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c0e009-3ded-4c05-b10a-db68ac83ee76 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:08:54.388844 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:08:54.396311 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c0e009-3ded-4c05-b10a-db68ac83ee76 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:08:54.434583 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7b46cd-a635-42d9-8243-3d60e5a6dc9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:08:54.435211 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7b46cd-a635-42d9-8243-3d60e5a6dc9b None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:08:54.435526 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7b46cd-a635-42d9-8243-3d60e5a6dc9b None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:08:54.435800 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7b46cd-a635-42d9-8243-3d60e5a6dc9b None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:08:54.436537 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7b46cd-a635-42d9-8243-3d60e5a6dc9b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:08:54.436792 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7b46cd-a635-42d9-8243-3d60e5a6dc9b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:08:54.438238 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7b46cd-a635-42d9-8243-3d60e5a6dc9b None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:08:54.450978 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 628/1243] 10.222.0.61 () {56 vars in 1109 bytes} [Mon Aug 22 19:08:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:08:54.456919 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fa9a88-c929-4419-9eee-6c0d490a243f None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:08:54.550551 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:08:54.579838 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fa9a88-c929-4419-9eee-6c0d490a243f None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:08:54.616222 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fab7a02-4abc-4a0e-a9f5-26a48344990d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:08:54.616863 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fab7a02-4abc-4a0e-a9f5-26a48344990d None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:08:54.617137 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fab7a02-4abc-4a0e-a9f5-26a48344990d None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:08:54.617405 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fab7a02-4abc-4a0e-a9f5-26a48344990d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:08:54.624713 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fab7a02-4abc-4a0e-a9f5-26a48344990d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:08:54.626172 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fab7a02-4abc-4a0e-a9f5-26a48344990d None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:08:54.637624 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-9fab7a02-4abc-4a0e-a9f5-26a48344990d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 19:08:54.637624 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:08:54.637624 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:08:54.637624 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:08:54.637624 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:08:54.637624 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:08:54.637624 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:08:54.637624 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:08:54.637624 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:08:54.637624 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:08:54.637624 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:08:54.637624 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:08:54.637624 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 19:08:54.637624 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 19:08:54.637624 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 19:08:54.638774 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 19:08:54.638774 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:08:54.638774 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:08:54.638774 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:08:54.638774 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:08:54.638774 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:08:54.638774 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:08:54.638774 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:08:54.638774 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:08:54.638774 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:08:54.638774 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:08:54.638774 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:08:54.638774 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:08:54.638774 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:08:54.638774 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:08:54.638774 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:08:54.638774 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:08:54.639977 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 19:08:54.639977 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 19:08:54.639977 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 19:08:54.639977 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 19:08:54.639977 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 19:08:54.639977 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 19:08:54.639977 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 19:08:54.639977 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:08:54.639977 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 616/1244] 10.222.0.61 () {56 vars in 1111 bytes} [Mon Aug 22 19:08:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:08:54.644949 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7b46cd-a635-42d9-8243-3d60e5a6dc9b None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:08:54.766792 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7b46cd-a635-42d9-8243-3d60e5a6dc9b None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:08:54.805957 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a124d2-8803-4d00-ac3c-8d0a918f0c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:08:54.807157 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a124d2-8803-4d00-ac3c-8d0a918f0c5e None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:08:54.807432 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a124d2-8803-4d00-ac3c-8d0a918f0c5e None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:08:54.807728 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a124d2-8803-4d00-ac3c-8d0a918f0c5e None admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:08:54.808426 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a124d2-8803-4d00-ac3c-8d0a918f0c5e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:08:54.808738 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a124d2-8803-4d00-ac3c-8d0a918f0c5e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:08:54.810159 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a124d2-8803-4d00-ac3c-8d0a918f0c5e None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:08:54.823623 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 629/1245] 10.222.0.61 () {56 vars in 1148 bytes} [Mon Aug 22 19:08:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:08:54.829059 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fab7a02-4abc-4a0e-a9f5-26a48344990d None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:08:54.932449 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fab7a02-4abc-4a0e-a9f5-26a48344990d None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:08:54.969633 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af5a5d1-c2bd-4c97-a1b2-75976d04e9ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:08:54.970244 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af5a5d1-c2bd-4c97-a1b2-75976d04e9ce None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:08:54.970491 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af5a5d1-c2bd-4c97-a1b2-75976d04e9ce None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:08:54.970763 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af5a5d1-c2bd-4c97-a1b2-75976d04e9ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:08:54.971495 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af5a5d1-c2bd-4c97-a1b2-75976d04e9ce None admin] RBAC: Adding query filter params (role.id=324bd04d55c34b61a5282c9f6ed73f46, scope.project.id=e07dc5ebdc504c6da706a2d21c0a3970, user.id=eef0f73297e04d62bc22acb790fad6ab) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:08:54.971839 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af5a5d1-c2bd-4c97-a1b2-75976d04e9ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:08:54.973249 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af5a5d1-c2bd-4c97-a1b2-75976d04e9ce None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:08:54.979230 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 617/1246] 10.222.0.61 () {56 vars in 1374 bytes} [Mon Aug 22 19:08:54 2022] GET /identity/v3/role_assignments?user.id=eef0f73297e04d62bc22acb790fad6ab&scope.project.id=e07dc5ebdc504c6da706a2d21c0a3970&role.id=324bd04d55c34b61a5282c9f6ed73f46 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 19:08:56.931360 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4787de27-a231-47d0-adb0-f96b0d3dae84 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:08:56.932072 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4787de27-a231-47d0-adb0-f96b0d3dae84 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:08:56.932072 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4787de27-a231-47d0-adb0-f96b0d3dae84 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:08:56.933295 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 630/1247] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:08:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:08:56.938965 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1744204-1043-452e-9ab4-de36bb2a1f0f None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:08:56.939502 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1744204-1043-452e-9ab4-de36bb2a1f0f None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:08:56.939502 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1744204-1043-452e-9ab4-de36bb2a1f0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:08:56.945874 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:08:56.960321 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-b1744204-1043-452e-9ab4-de36bb2a1f0f None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:08:56.961624 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-b1744204-1043-452e-9ab4-de36bb2a1f0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:08:57.033975 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 618/1248] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:08:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 22 19:08:57.300520 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4787de27-a231-47d0-adb0-f96b0d3dae84 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:08:57.320533 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:08:57.422850 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4787de27-a231-47d0-adb0-f96b0d3dae84 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:08:57.461632 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0664be7-8160-4666-91ca-a720ae2c272f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:08:57.463079 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0664be7-8160-4666-91ca-a720ae2c272f None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:08:57.463529 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0664be7-8160-4666-91ca-a720ae2c272f None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:08:57.463976 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0664be7-8160-4666-91ca-a720ae2c272f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:08:57.476637 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0664be7-8160-4666-91ca-a720ae2c272f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 19:08:57.482776 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-f0664be7-8160-4666-91ca-a720ae2c272f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 19:08:57.482776 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:08:57.482776 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:08:57.482776 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:08:57.482776 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:08:57.482776 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:08:57.482776 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:08:57.482776 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:08:57.482776 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:08:57.482776 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:08:57.482776 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:08:57.482776 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:08:57.482776 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 19:08:57.482776 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 19:08:57.482776 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 19:08:57.482776 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 19:08:57.483849 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 19:08:57.483849 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 19:08:57.483849 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 19:08:57.483849 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 19:08:57.483849 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:08:57.483849 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:08:57.483849 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:08:57.483849 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:08:57.483849 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:08:57.483849 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:08:57.483849 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:08:57.483849 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:08:57.483849 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:08:57.483849 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:08:57.483849 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:08:57.483849 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:08:57.484713 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:08:57.484713 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:08:57.484713 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:08:57.484713 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:08:57.484713 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 19:08:57.484713 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 19:08:57.484713 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 19:08:57.484713 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 19:08:57.484713 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 19:08:57.484713 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 19:08:57.484713 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 19:08:57.484713 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:08:57.484713 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 631/1249] 10.222.0.61 () {56 vars in 1096 bytes} [Mon Aug 22 19:08:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:08:57.490379 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1744204-1043-452e-9ab4-de36bb2a1f0f None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:08:57.608910 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1744204-1043-452e-9ab4-de36bb2a1f0f None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:08:57.641535 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:08:57.647204 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5897a71-a21d-4b30-ac44-88645e2caf56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:08:57.647849 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5897a71-a21d-4b30-ac44-88645e2caf56 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:08:57.648127 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5897a71-a21d-4b30-ac44-88645e2caf56 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:08:57.648399 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5897a71-a21d-4b30-ac44-88645e2caf56 None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:08:57.649096 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5897a71-a21d-4b30-ac44-88645e2caf56 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:08:57.649436 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5897a71-a21d-4b30-ac44-88645e2caf56 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:08:57.651685 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5897a71-a21d-4b30-ac44-88645e2caf56 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:08:57.661282 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 619/1250] 10.222.0.61 () {56 vars in 1099 bytes} [Mon Aug 22 19:08:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 19:08:57.667173 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0664be7-8160-4666-91ca-a720ae2c272f None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:08:57.754480 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:08:57.781090 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0664be7-8160-4666-91ca-a720ae2c272f None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:08:57.817949 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381bca4a-ea62-4c39-bfd4-f0da4e8a8275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:08:57.818616 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381bca4a-ea62-4c39-bfd4-f0da4e8a8275 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:08:57.818860 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381bca4a-ea62-4c39-bfd4-f0da4e8a8275 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:08:57.819128 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381bca4a-ea62-4c39-bfd4-f0da4e8a8275 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:08:57.829207 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381bca4a-ea62-4c39-bfd4-f0da4e8a8275 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:08:57.830631 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381bca4a-ea62-4c39-bfd4-f0da4e8a8275 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:08:57.845382 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-381bca4a-ea62-4c39-bfd4-f0da4e8a8275 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:08:57.845382 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:08:57.845382 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:08:57.845382 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:08:57.845382 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:08:57.845382 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:08:57.845382 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:08:57.845382 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:08:57.845382 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:08:57.845382 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:08:57.845382 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:08:57.845382 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:08:57.845382 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 19:08:57.845382 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 19:08:57.845382 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 19:08:57.845382 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 19:08:57.846543 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:08:57.846543 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:08:57.846543 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:08:57.846543 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:08:57.846543 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:08:57.846543 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:08:57.846543 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:08:57.846543 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:08:57.846543 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:08:57.846543 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:08:57.846543 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:08:57.846543 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:08:57.846543 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:08:57.846543 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:08:57.846543 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:08:57.846543 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:08:57.846543 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 19:08:57.847625 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 19:08:57.847625 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:08:57.847625 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:08:57.851732 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 632/1251] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:08:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 19:08:57.852746 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5897a71-a21d-4b30-ac44-88645e2caf56 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:08:57.961924 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5897a71-a21d-4b30-ac44-88645e2caf56 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:08:58.000803 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceffe46-9515-4f62-b967-5301071a8ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:08:58.001437 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceffe46-9515-4f62-b967-5301071a8ca8 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:08:58.001829 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceffe46-9515-4f62-b967-5301071a8ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:08:58.002109 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceffe46-9515-4f62-b967-5301071a8ca8 None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:08:58.002925 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ceffe46-9515-4f62-b967-5301071a8ca8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:08:58.003277 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ceffe46-9515-4f62-b967-5301071a8ca8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:08:58.004950 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ceffe46-9515-4f62-b967-5301071a8ca8 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:08:58.019705 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 620/1252] 10.222.0.61 () {56 vars in 1109 bytes} [Mon Aug 22 19:08:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 19:08:58.023710 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381bca4a-ea62-4c39-bfd4-f0da4e8a8275 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:08:58.128242 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381bca4a-ea62-4c39-bfd4-f0da4e8a8275 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:08:58.156806 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:08:58.165198 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ac1950-69d8-448b-9d8d-e171f50b1b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:08:58.166486 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ac1950-69d8-448b-9d8d-e171f50b1b70 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:08:58.166855 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ac1950-69d8-448b-9d8d-e171f50b1b70 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:08:58.167198 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ac1950-69d8-448b-9d8d-e171f50b1b70 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:08:58.176353 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ac1950-69d8-448b-9d8d-e171f50b1b70 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:08:58.177795 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ac1950-69d8-448b-9d8d-e171f50b1b70 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:08:58.189713 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-a1ac1950-69d8-448b-9d8d-e171f50b1b70 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 22 19:08:58.189713 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:08:58.189713 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:08:58.189713 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:08:58.189713 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:08:58.189713 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:08:58.189713 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:08:58.189713 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:08:58.189713 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:08:58.189713 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:08:58.189713 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:08:58.189713 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:08:58.189713 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 19:08:58.189713 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 19:08:58.189713 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 19:08:58.189713 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 19:08:58.190894 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:08:58.190894 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:08:58.190894 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 19:08:58.190894 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 19:08:58.190894 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 19:08:58.190894 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 19:08:58.190894 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:08:58.190894 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:08:58.190894 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:08:58.190894 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:08:58.190894 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:08:58.190894 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:08:58.190894 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:08:58.190894 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:08:58.190894 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:08:58.190894 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:08:58.190894 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:08:58.190894 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:08:58.191985 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:08:58.191985 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:08:58.191985 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 19:08:58.191985 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 19:08:58.191985 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 19:08:58.191985 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 19:08:58.191985 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 19:08:58.191985 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 19:08:58.191985 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 22 19:08:58.191985 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:08:58.191985 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 633/1253] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:08:58 2022] GET /identity/v3/users/placement => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:08:58.196857 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceffe46-9515-4f62-b967-5301071a8ca8 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:08:58.260236 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:08:58.309487 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceffe46-9515-4f62-b967-5301071a8ca8 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:08:58.348481 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4301893b-93a8-49cf-8cfc-1647bceba0ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:08:58.349179 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4301893b-93a8-49cf-8cfc-1647bceba0ab None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:08:58.349474 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4301893b-93a8-49cf-8cfc-1647bceba0ab None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:08:58.349779 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4301893b-93a8-49cf-8cfc-1647bceba0ab None admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:08:58.350697 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4301893b-93a8-49cf-8cfc-1647bceba0ab None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:08:58.351077 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4301893b-93a8-49cf-8cfc-1647bceba0ab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:08:58.352759 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4301893b-93a8-49cf-8cfc-1647bceba0ab None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:08:58.371164 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 621/1254] 10.222.0.61 () {56 vars in 1143 bytes} [Mon Aug 22 19:08:58 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:08:58.377056 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ac1950-69d8-448b-9d8d-e171f50b1b70 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:08:58.492944 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ac1950-69d8-448b-9d8d-e171f50b1b70 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:08:58.536471 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f49acc-161e-44d1-9916-f834672b201d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:08:58.536985 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f49acc-161e-44d1-9916-f834672b201d None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:08:58.537389 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f49acc-161e-44d1-9916-f834672b201d None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:08:58.537879 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f49acc-161e-44d1-9916-f834672b201d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:08:58.611359 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:08:58.627534 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f49acc-161e-44d1-9916-f834672b201d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:08:58.629480 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f49acc-161e-44d1-9916-f834672b201d None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:08:58.745702 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 634/1255] 10.222.0.61 () {58 vars in 1282 bytes} [Mon Aug 22 19:08:58 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 22 19:08:58.751845 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4301893b-93a8-49cf-8cfc-1647bceba0ab None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:08:58.859884 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4301893b-93a8-49cf-8cfc-1647bceba0ab None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:08:58.875244 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:08:58.898751 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813eba5a-5ff1-46ab-9319-02e438a93a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:08:58.899464 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813eba5a-5ff1-46ab-9319-02e438a93a07 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:08:58.899882 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813eba5a-5ff1-46ab-9319-02e438a93a07 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:08:58.900158 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813eba5a-5ff1-46ab-9319-02e438a93a07 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:08:58.911396 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813eba5a-5ff1-46ab-9319-02e438a93a07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:08:58.913258 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813eba5a-5ff1-46ab-9319-02e438a93a07 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:08:58.930378 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-813eba5a-5ff1-46ab-9319-02e438a93a07 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:08:58.930378 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:08:58.930378 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:08:58.930378 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:08:58.930378 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:08:58.930378 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:08:58.930378 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:08:58.930378 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:08:58.930378 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:08:58.930378 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:08:58.930378 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:08:58.930378 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:08:58.930378 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 19:08:58.930378 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 19:08:58.930378 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 19:08:58.930378 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 19:08:58.932073 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:08:58.932073 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:08:58.932073 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:08:58.932073 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:08:58.932073 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:08:58.932073 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:08:58.932073 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:08:58.932073 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:08:58.932073 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:08:58.932073 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:08:58.932073 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:08:58.932073 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:08:58.932073 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:08:58.932073 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:08:58.932073 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:08:58.932073 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:08:58.932073 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 19:08:58.937770 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 19:08:58.937770 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:08:58.937770 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:08:58.937770 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 622/1256] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:08:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:08:58.938292 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f49acc-161e-44d1-9916-f834672b201d None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:08:59.035686 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:08:59.049865 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f49acc-161e-44d1-9916-f834672b201d None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:08:59.088924 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bb7691-4785-4bfc-9567-76e4fe90fbb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:08:59.089648 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bb7691-4785-4bfc-9567-76e4fe90fbb4 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:08:59.089924 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bb7691-4785-4bfc-9567-76e4fe90fbb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:08:59.090193 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bb7691-4785-4bfc-9567-76e4fe90fbb4 None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:08:59.090832 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bb7691-4785-4bfc-9567-76e4fe90fbb4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:08:59.091129 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bb7691-4785-4bfc-9567-76e4fe90fbb4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:08:59.093792 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bb7691-4785-4bfc-9567-76e4fe90fbb4 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:08:59.106046 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 635/1257] 10.222.0.61 () {56 vars in 1109 bytes} [Mon Aug 22 19:08:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:08:59.113503 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813eba5a-5ff1-46ab-9319-02e438a93a07 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:08:59.231274 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813eba5a-5ff1-46ab-9319-02e438a93a07 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:08:59.269938 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db87cd5-9dcd-4816-a02f-c1c6050a2591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:08:59.270672 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db87cd5-9dcd-4816-a02f-c1c6050a2591 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:08:59.270998 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db87cd5-9dcd-4816-a02f-c1c6050a2591 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:08:59.271275 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db87cd5-9dcd-4816-a02f-c1c6050a2591 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:08:59.278968 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db87cd5-9dcd-4816-a02f-c1c6050a2591 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:08:59.280646 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db87cd5-9dcd-4816-a02f-c1c6050a2591 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:08:59.292028 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-4db87cd5-9dcd-4816-a02f-c1c6050a2591 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 19:08:59.292028 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:08:59.292028 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:08:59.292028 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:08:59.292028 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:08:59.292028 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:08:59.292028 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:08:59.292028 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:08:59.292028 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:08:59.292028 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:08:59.292028 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:08:59.292028 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:08:59.292028 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 19:08:59.292028 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 19:08:59.292028 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 19:08:59.293239 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 19:08:59.293239 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:08:59.293239 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:08:59.293239 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:08:59.293239 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:08:59.293239 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:08:59.293239 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:08:59.293239 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:08:59.293239 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:08:59.293239 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:08:59.293239 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:08:59.293239 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:08:59.293239 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:08:59.293239 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:08:59.293239 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:08:59.293239 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:08:59.293239 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:08:59.294308 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 19:08:59.294308 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 19:08:59.294308 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 19:08:59.294308 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 19:08:59.294308 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 19:08:59.294308 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 19:08:59.294308 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 19:08:59.294308 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:08:59.294308 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 623/1258] 10.222.0.61 () {56 vars in 1111 bytes} [Mon Aug 22 19:08:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:08:59.299662 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bb7691-4785-4bfc-9567-76e4fe90fbb4 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:08:59.412379 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bb7691-4785-4bfc-9567-76e4fe90fbb4 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:08:59.451082 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2800aa68-c9ea-4bc4-a9c7-bf6931ab58c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:08:59.451757 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2800aa68-c9ea-4bc4-a9c7-bf6931ab58c8 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:08:59.452183 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2800aa68-c9ea-4bc4-a9c7-bf6931ab58c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:08:59.452460 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2800aa68-c9ea-4bc4-a9c7-bf6931ab58c8 None admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:08:59.453173 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2800aa68-c9ea-4bc4-a9c7-bf6931ab58c8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:08:59.453498 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2800aa68-c9ea-4bc4-a9c7-bf6931ab58c8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:08:59.455074 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2800aa68-c9ea-4bc4-a9c7-bf6931ab58c8 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:08:59.469111 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 636/1259] 10.222.0.61 () {56 vars in 1148 bytes} [Mon Aug 22 19:08:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:08:59.475727 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db87cd5-9dcd-4816-a02f-c1c6050a2591 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:08:59.506110 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:08:59.592389 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db87cd5-9dcd-4816-a02f-c1c6050a2591 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:08:59.628531 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84becff-aeeb-4fe6-a315-bbc9525a5fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:08:59.629143 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84becff-aeeb-4fe6-a315-bbc9525a5fdb None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:08:59.629423 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84becff-aeeb-4fe6-a315-bbc9525a5fdb None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:08:59.629689 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84becff-aeeb-4fe6-a315-bbc9525a5fdb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:08:59.630416 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84becff-aeeb-4fe6-a315-bbc9525a5fdb None admin] RBAC: Adding query filter params (role.id=163c30c3b3c246beb849ce176f02d9da, scope.project.id=e07dc5ebdc504c6da706a2d21c0a3970, user.id=eef0f73297e04d62bc22acb790fad6ab) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:08:59.630718 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84becff-aeeb-4fe6-a315-bbc9525a5fdb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:08:59.632176 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84becff-aeeb-4fe6-a315-bbc9525a5fdb None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:08:59.638614 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 624/1260] 10.222.0.61 () {56 vars in 1374 bytes} [Mon Aug 22 19:08:59 2022] GET /identity/v3/role_assignments?user.id=eef0f73297e04d62bc22acb790fad6ab&scope.project.id=e07dc5ebdc504c6da706a2d21c0a3970&role.id=163c30c3b3c246beb849ce176f02d9da => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:09:01.501391 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffe07d1-8846-4b90-b411-bebd2abbfadb None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:09:01.501917 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffe07d1-8846-4b90-b411-bebd2abbfadb None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:09:01.501917 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffe07d1-8846-4b90-b411-bebd2abbfadb None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:09:01.503273 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 637/1261] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:09:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:09:01.508241 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676627b4-f9a1-4391-ae9c-61bb0e8c5115 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:09:01.508623 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676627b4-f9a1-4391-ae9c-61bb0e8c5115 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:09:01.508778 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676627b4-f9a1-4391-ae9c-61bb0e8c5115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:09:01.527228 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-676627b4-f9a1-4391-ae9c-61bb0e8c5115 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:09:01.528508 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-676627b4-f9a1-4391-ae9c-61bb0e8c5115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:09:01.596783 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 625/1262] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:09:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 19:09:01.857697 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffe07d1-8846-4b90-b411-bebd2abbfadb None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:09:01.868776 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:09:01.958803 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffe07d1-8846-4b90-b411-bebd2abbfadb None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:09:01.993491 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fee6f28-7650-445d-8643-97c9b1e52188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:09:01.994118 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fee6f28-7650-445d-8643-97c9b1e52188 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:09:01.994398 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fee6f28-7650-445d-8643-97c9b1e52188 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:09:01.994671 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fee6f28-7650-445d-8643-97c9b1e52188 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:09:02.004621 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fee6f28-7650-445d-8643-97c9b1e52188 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 19:09:02.009367 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-1fee6f28-7650-445d-8643-97c9b1e52188 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 19:09:02.009367 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:09:02.009367 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:09:02.009367 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:09:02.009367 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:09:02.009367 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:09:02.009367 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:09:02.009367 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:09:02.009367 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:09:02.009367 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:09:02.009367 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:09:02.009367 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:09:02.009367 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 19:09:02.009367 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 19:09:02.009367 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 19:09:02.009367 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 19:09:02.010346 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 19:09:02.010346 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 19:09:02.010346 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 19:09:02.010346 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 19:09:02.010346 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:09:02.010346 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:09:02.010346 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:09:02.010346 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:09:02.010346 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:09:02.010346 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:09:02.010346 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:09:02.010346 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:09:02.010346 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:09:02.010346 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:09:02.010346 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:09:02.010346 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:09:02.011229 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:09:02.011229 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:09:02.011229 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:09:02.011229 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:09:02.011229 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 19:09:02.011229 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 19:09:02.011229 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 19:09:02.011229 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 19:09:02.011229 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 19:09:02.011229 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 19:09:02.011229 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 19:09:02.011229 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:09:02.011229 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 638/1263] 10.222.0.61 () {56 vars in 1096 bytes} [Mon Aug 22 19:09:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:09:02.016430 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676627b4-f9a1-4391-ae9c-61bb0e8c5115 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:09:02.034415 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:09:02.114238 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676627b4-f9a1-4391-ae9c-61bb0e8c5115 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:09:02.146840 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22a5ed4-0bc1-4860-b42e-c57413b73bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:09:02.147435 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22a5ed4-0bc1-4860-b42e-c57413b73bdb None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:09:02.147738 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22a5ed4-0bc1-4860-b42e-c57413b73bdb None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:09:02.148005 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22a5ed4-0bc1-4860-b42e-c57413b73bdb None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:09:02.148660 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22a5ed4-0bc1-4860-b42e-c57413b73bdb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:09:02.148952 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22a5ed4-0bc1-4860-b42e-c57413b73bdb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:09:02.150904 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22a5ed4-0bc1-4860-b42e-c57413b73bdb None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:09:02.158759 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 626/1264] 10.222.0.61 () {56 vars in 1099 bytes} [Mon Aug 22 19:09:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:09:02.164378 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fee6f28-7650-445d-8643-97c9b1e52188 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:09:02.228920 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:09:02.261626 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fee6f28-7650-445d-8643-97c9b1e52188 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:09:02.294465 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169d6996-dc6f-4f8a-bac7-e6ab064fb4a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:09:02.295081 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169d6996-dc6f-4f8a-bac7-e6ab064fb4a1 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:09:02.295357 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169d6996-dc6f-4f8a-bac7-e6ab064fb4a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:09:02.295627 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169d6996-dc6f-4f8a-bac7-e6ab064fb4a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:09:02.305322 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169d6996-dc6f-4f8a-bac7-e6ab064fb4a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:09:02.306752 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169d6996-dc6f-4f8a-bac7-e6ab064fb4a1 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:09:02.320726 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-169d6996-dc6f-4f8a-bac7-e6ab064fb4a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:09:02.320726 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:09:02.320726 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:09:02.320726 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:09:02.320726 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:09:02.320726 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:09:02.320726 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:09:02.320726 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:09:02.320726 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:09:02.320726 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:09:02.320726 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:09:02.320726 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:09:02.320726 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 19:09:02.320726 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 19:09:02.320726 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 19:09:02.320726 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 19:09:02.322319 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:09:02.322319 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:09:02.322319 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:09:02.322319 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:09:02.322319 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:09:02.322319 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:09:02.322319 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:09:02.322319 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:09:02.322319 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:09:02.322319 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:09:02.322319 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:09:02.322319 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:09:02.322319 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:09:02.322319 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:09:02.322319 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:09:02.322319 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:09:02.322319 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 19:09:02.323246 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 19:09:02.323246 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:09:02.323246 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:09:02.323246 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 639/1265] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:09:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:09:02.328002 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22a5ed4-0bc1-4860-b42e-c57413b73bdb None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:09:02.406829 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:09:02.425368 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22a5ed4-0bc1-4860-b42e-c57413b73bdb None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:09:02.457700 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b2c2c7-a3fd-4fd6-ba24-d58abe7375ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:09:02.458295 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b2c2c7-a3fd-4fd6-ba24-d58abe7375ca None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:09:02.458571 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b2c2c7-a3fd-4fd6-ba24-d58abe7375ca None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:09:02.458840 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b2c2c7-a3fd-4fd6-ba24-d58abe7375ca None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:09:02.459482 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b2c2c7-a3fd-4fd6-ba24-d58abe7375ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:09:02.459820 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b2c2c7-a3fd-4fd6-ba24-d58abe7375ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:09:02.461167 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b2c2c7-a3fd-4fd6-ba24-d58abe7375ca None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:09:02.473082 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 627/1266] 10.222.0.61 () {56 vars in 1109 bytes} [Mon Aug 22 19:09:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:09:02.478706 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169d6996-dc6f-4f8a-bac7-e6ab064fb4a1 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:09:02.580413 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169d6996-dc6f-4f8a-bac7-e6ab064fb4a1 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:09:02.599590 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:09:02.613992 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cd9943-b45a-41de-848e-4c33f038ff20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:09:02.614613 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cd9943-b45a-41de-848e-4c33f038ff20 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:09:02.614885 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cd9943-b45a-41de-848e-4c33f038ff20 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:09:02.615154 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cd9943-b45a-41de-848e-4c33f038ff20 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:09:02.623073 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cd9943-b45a-41de-848e-4c33f038ff20 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:09:02.624522 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cd9943-b45a-41de-848e-4c33f038ff20 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:09:02.635269 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-31cd9943-b45a-41de-848e-4c33f038ff20 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 22 19:09:02.635269 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:09:02.635269 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:09:02.635269 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:09:02.635269 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:09:02.635269 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:09:02.635269 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:09:02.635269 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:09:02.635269 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:09:02.635269 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:09:02.635269 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:09:02.635269 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:09:02.635269 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 19:09:02.635269 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 19:09:02.635269 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 19:09:02.635269 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 19:09:02.636407 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:09:02.636407 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:09:02.636407 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 19:09:02.636407 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 19:09:02.636407 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 19:09:02.636407 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 19:09:02.636407 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:09:02.636407 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:09:02.636407 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:09:02.636407 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:09:02.636407 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:09:02.636407 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:09:02.636407 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:09:02.636407 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:09:02.636407 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:09:02.636407 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:09:02.636407 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:09:02.636407 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:09:02.637581 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:09:02.637581 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:09:02.637581 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 19:09:02.637581 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 19:09:02.637581 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 19:09:02.637581 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 19:09:02.637581 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 19:09:02.637581 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 19:09:02.637581 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 22 19:09:02.637581 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:09:02.637581 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 640/1267] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:09:02 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 19:09:02.642126 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b2c2c7-a3fd-4fd6-ba24-d58abe7375ca None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:09:02.739637 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b2c2c7-a3fd-4fd6-ba24-d58abe7375ca None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:09:02.772016 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5ac48f-3c0b-431c-8206-c5ae884f2b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:09:02.772615 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5ac48f-3c0b-431c-8206-c5ae884f2b98 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:09:02.772892 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5ac48f-3c0b-431c-8206-c5ae884f2b98 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:09:02.773162 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5ac48f-3c0b-431c-8206-c5ae884f2b98 None admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:09:02.773883 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5ac48f-3c0b-431c-8206-c5ae884f2b98 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:09:02.774187 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5ac48f-3c0b-431c-8206-c5ae884f2b98 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:09:02.776179 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5ac48f-3c0b-431c-8206-c5ae884f2b98 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:09:02.784536 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:09:02.793377 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 628/1268] 10.222.0.61 () {56 vars in 1143 bytes} [Mon Aug 22 19:09:02 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:09:02.798916 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cd9943-b45a-41de-848e-4c33f038ff20 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:09:02.895490 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cd9943-b45a-41de-848e-4c33f038ff20 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:09:02.927512 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799e6f39-df7b-4804-8284-120ea089ec5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:09:02.928152 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799e6f39-df7b-4804-8284-120ea089ec5c None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:09:02.928427 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799e6f39-df7b-4804-8284-120ea089ec5c None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:09:02.928694 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799e6f39-df7b-4804-8284-120ea089ec5c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:09:02.939416 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799e6f39-df7b-4804-8284-120ea089ec5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:09:02.940879 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799e6f39-df7b-4804-8284-120ea089ec5c None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:09:02.955036 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-799e6f39-df7b-4804-8284-120ea089ec5c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:09:02.955036 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:09:02.955036 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:09:02.955036 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:09:02.955036 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:09:02.955036 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:09:02.955036 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:09:02.955036 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:09:02.955036 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:09:02.955036 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:09:02.955036 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:09:02.955036 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:09:02.955036 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 19:09:02.955036 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 19:09:02.955036 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 19:09:02.955036 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 19:09:02.956191 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:09:02.956191 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:09:02.956191 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:09:02.956191 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:09:02.956191 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:09:02.956191 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:09:02.956191 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:09:02.956191 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:09:02.956191 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:09:02.956191 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:09:02.956191 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:09:02.956191 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:09:02.956191 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:09:02.956191 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:09:02.956191 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:09:02.956191 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:09:02.956191 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 19:09:02.957259 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 19:09:02.957259 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:09:02.957259 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:09:02.957259 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 641/1269] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:09:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:09:02.961930 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5ac48f-3c0b-431c-8206-c5ae884f2b98 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:09:03.059592 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5ac48f-3c0b-431c-8206-c5ae884f2b98 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:09:03.092967 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52df4e19-e95f-44b5-ab7e-5d4a9314cc0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:09:03.093564 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52df4e19-e95f-44b5-ab7e-5d4a9314cc0f None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:09:03.093842 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52df4e19-e95f-44b5-ab7e-5d4a9314cc0f None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:09:03.094112 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52df4e19-e95f-44b5-ab7e-5d4a9314cc0f None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:09:03.094745 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52df4e19-e95f-44b5-ab7e-5d4a9314cc0f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:09:03.095044 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52df4e19-e95f-44b5-ab7e-5d4a9314cc0f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:09:03.096490 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52df4e19-e95f-44b5-ab7e-5d4a9314cc0f None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:09:03.107244 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 629/1270] 10.222.0.61 () {56 vars in 1109 bytes} [Mon Aug 22 19:09:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:09:03.113256 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799e6f39-df7b-4804-8284-120ea089ec5c None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:09:03.144244 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:09:03.210810 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799e6f39-df7b-4804-8284-120ea089ec5c None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:09:03.242998 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf5723a-830a-494e-8107-b5b7903f8461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:09:03.244164 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf5723a-830a-494e-8107-b5b7903f8461 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:09:03.244437 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf5723a-830a-494e-8107-b5b7903f8461 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:09:03.244703 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf5723a-830a-494e-8107-b5b7903f8461 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:09:03.251000 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf5723a-830a-494e-8107-b5b7903f8461 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:09:03.252444 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf5723a-830a-494e-8107-b5b7903f8461 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:09:03.263770 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-edf5723a-830a-494e-8107-b5b7903f8461 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 19:09:03.263770 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:09:03.263770 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:09:03.263770 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:09:03.263770 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:09:03.263770 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:09:03.263770 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:09:03.263770 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:09:03.263770 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:09:03.263770 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:09:03.263770 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:09:03.263770 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:09:03.263770 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 19:09:03.263770 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 19:09:03.263770 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 19:09:03.264786 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 19:09:03.264786 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:09:03.264786 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:09:03.264786 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:09:03.264786 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:09:03.264786 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:09:03.264786 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:09:03.264786 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:09:03.264786 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:09:03.264786 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:09:03.264786 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:09:03.264786 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:09:03.264786 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:09:03.264786 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:09:03.264786 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:09:03.264786 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:09:03.264786 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:09:03.265681 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 19:09:03.265681 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 19:09:03.265681 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 19:09:03.265681 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 19:09:03.265681 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 19:09:03.265681 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 19:09:03.265681 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 19:09:03.265681 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:09:03.265681 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 642/1271] 10.222.0.61 () {56 vars in 1111 bytes} [Mon Aug 22 19:09:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:09:03.270725 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52df4e19-e95f-44b5-ab7e-5d4a9314cc0f None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:09:03.328791 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:09:03.370415 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52df4e19-e95f-44b5-ab7e-5d4a9314cc0f None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:09:03.402707 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd02a95e-95c1-4118-93f0-fd6116e2b159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:09:03.403296 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd02a95e-95c1-4118-93f0-fd6116e2b159 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:09:03.403568 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd02a95e-95c1-4118-93f0-fd6116e2b159 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:09:03.403865 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd02a95e-95c1-4118-93f0-fd6116e2b159 None admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:09:03.404546 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd02a95e-95c1-4118-93f0-fd6116e2b159 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:09:03.404842 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd02a95e-95c1-4118-93f0-fd6116e2b159 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:09:03.406234 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd02a95e-95c1-4118-93f0-fd6116e2b159 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:09:03.417715 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 630/1272] 10.222.0.61 () {56 vars in 1148 bytes} [Mon Aug 22 19:09:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 19:09:03.423677 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf5723a-830a-494e-8107-b5b7903f8461 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:09:03.513006 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:09:03.522158 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf5723a-830a-494e-8107-b5b7903f8461 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:09:03.565731 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926771bd-0f83-4cb2-8e40-1700aa6793ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:09:03.566388 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926771bd-0f83-4cb2-8e40-1700aa6793ee None admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:09:03.566666 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926771bd-0f83-4cb2-8e40-1700aa6793ee None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:09:03.566940 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926771bd-0f83-4cb2-8e40-1700aa6793ee None admin] PATH_INFO: `/v3/projects/e07dc5ebdc504c6da706a2d21c0a3970/users/eef0f73297e04d62bc22acb790fad6ab/roles/163c30c3b3c246beb849ce176f02d9da` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:09:03.568834 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926771bd-0f83-4cb2-8e40-1700aa6793ee None admin] RBAC: Authorizing `identity:create_grant(project_id=e07dc5ebdc504c6da706a2d21c0a3970, user_id=eef0f73297e04d62bc22acb790fad6ab, role_id=163c30c3b3c246beb849ce176f02d9da)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:09:03.570258 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926771bd-0f83-4cb2-8e40-1700aa6793ee None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:09:03.582821 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 643/1273] 10.222.0.61 () {58 vars in 1439 bytes} [Mon Aug 22 19:09:03 2022] PUT /identity/v3/projects/e07dc5ebdc504c6da706a2d21c0a3970/users/eef0f73297e04d62bc22acb790fad6ab/roles/163c30c3b3c246beb849ce176f02d9da => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:09:05.447735 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac97eb6-7f0d-4bc1-99a7-572953aad4c4 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:09:05.448253 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac97eb6-7f0d-4bc1-99a7-572953aad4c4 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:09:05.448365 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac97eb6-7f0d-4bc1-99a7-572953aad4c4 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:09:05.449645 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 631/1274] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:09:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:09:05.454693 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db3cfeb-e0c9-4ea8-bae7-e1b194fa6499 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:09:05.455072 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db3cfeb-e0c9-4ea8-bae7-e1b194fa6499 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:09:05.455244 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db3cfeb-e0c9-4ea8-bae7-e1b194fa6499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:09:05.473435 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-6db3cfeb-e0c9-4ea8-bae7-e1b194fa6499 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:09:05.474626 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-6db3cfeb-e0c9-4ea8-bae7-e1b194fa6499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:09:05.542547 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 644/1275] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:09:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 19:09:05.813609 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac97eb6-7f0d-4bc1-99a7-572953aad4c4 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:09:05.911535 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac97eb6-7f0d-4bc1-99a7-572953aad4c4 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:09:05.928519 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:09:05.945341 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f45c95-1243-4770-b9f7-27a658f94b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:09:05.945963 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f45c95-1243-4770-b9f7-27a658f94b0b None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:09:05.946263 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f45c95-1243-4770-b9f7-27a658f94b0b None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:09:05.946532 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f45c95-1243-4770-b9f7-27a658f94b0b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:09:05.956254 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f45c95-1243-4770-b9f7-27a658f94b0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 19:09:05.960916 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-d9f45c95-1243-4770-b9f7-27a658f94b0b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 19:09:05.960916 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:09:05.960916 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:09:05.960916 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:09:05.960916 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:09:05.960916 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:09:05.960916 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:09:05.960916 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:09:05.960916 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:09:05.960916 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:09:05.960916 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:09:05.960916 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:09:05.960916 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 19:09:05.960916 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 19:09:05.960916 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 19:09:05.960916 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 19:09:05.962018 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 19:09:05.962018 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 19:09:05.962018 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 19:09:05.962018 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 19:09:05.962018 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:09:05.962018 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:09:05.962018 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:09:05.962018 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:09:05.962018 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:09:05.962018 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:09:05.962018 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:09:05.962018 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:09:05.962018 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:09:05.962018 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:09:05.962018 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:09:05.962018 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:09:05.963201 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:09:05.963201 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:09:05.963201 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:09:05.963201 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:09:05.963201 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 19:09:05.963201 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 19:09:05.963201 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 19:09:05.963201 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 19:09:05.963201 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 19:09:05.963201 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 19:09:05.963201 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 19:09:05.963201 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:09:05.963201 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 632/1276] 10.222.0.61 () {56 vars in 1096 bytes} [Mon Aug 22 19:09:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:09:05.967848 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db3cfeb-e0c9-4ea8-bae7-e1b194fa6499 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:09:06.047767 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:09:06.066651 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db3cfeb-e0c9-4ea8-bae7-e1b194fa6499 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:09:06.100669 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc55f9a7-697a-4bc7-ab59-9c1862d2b0f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:09:06.101273 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc55f9a7-697a-4bc7-ab59-9c1862d2b0f4 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:09:06.101547 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc55f9a7-697a-4bc7-ab59-9c1862d2b0f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:09:06.101815 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc55f9a7-697a-4bc7-ab59-9c1862d2b0f4 None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:09:06.103040 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc55f9a7-697a-4bc7-ab59-9c1862d2b0f4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:09:06.103349 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc55f9a7-697a-4bc7-ab59-9c1862d2b0f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:09:06.104834 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc55f9a7-697a-4bc7-ab59-9c1862d2b0f4 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:09:06.113094 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 645/1277] 10.222.0.61 () {56 vars in 1099 bytes} [Mon Aug 22 19:09:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:09:06.119439 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f45c95-1243-4770-b9f7-27a658f94b0b None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:09:06.221131 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f45c95-1243-4770-b9f7-27a658f94b0b None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:09:06.253959 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50793d8e-2b3d-4c63-b1b9-34d7c795d646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:09:06.254576 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50793d8e-2b3d-4c63-b1b9-34d7c795d646 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:09:06.254852 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50793d8e-2b3d-4c63-b1b9-34d7c795d646 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:09:06.255122 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50793d8e-2b3d-4c63-b1b9-34d7c795d646 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:09:06.265343 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50793d8e-2b3d-4c63-b1b9-34d7c795d646 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:09:06.266770 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50793d8e-2b3d-4c63-b1b9-34d7c795d646 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:09:06.281198 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-50793d8e-2b3d-4c63-b1b9-34d7c795d646 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:09:06.281198 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:09:06.281198 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:09:06.281198 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:09:06.281198 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:09:06.281198 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:09:06.281198 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:09:06.281198 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:09:06.281198 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:09:06.281198 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:09:06.281198 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:09:06.281198 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:09:06.281198 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 19:09:06.281198 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 19:09:06.281198 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 19:09:06.281198 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 19:09:06.282273 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:09:06.282273 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:09:06.282273 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:09:06.282273 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:09:06.282273 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:09:06.282273 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:09:06.282273 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:09:06.282273 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:09:06.282273 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:09:06.282273 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:09:06.282273 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:09:06.282273 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:09:06.282273 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:09:06.282273 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:09:06.282273 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:09:06.282273 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:09:06.282273 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 19:09:06.283542 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 19:09:06.283542 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:09:06.283542 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:09:06.283542 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 633/1278] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:09:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:09:06.288175 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc55f9a7-697a-4bc7-ab59-9c1862d2b0f4 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:09:06.385544 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc55f9a7-697a-4bc7-ab59-9c1862d2b0f4 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:09:06.418243 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91b653d-1ac0-4e25-b987-d0cb16dcacc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:09:06.418844 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91b653d-1ac0-4e25-b987-d0cb16dcacc9 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:09:06.419121 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91b653d-1ac0-4e25-b987-d0cb16dcacc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:09:06.419390 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91b653d-1ac0-4e25-b987-d0cb16dcacc9 None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:09:06.420050 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91b653d-1ac0-4e25-b987-d0cb16dcacc9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:09:06.420350 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91b653d-1ac0-4e25-b987-d0cb16dcacc9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:09:06.421702 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91b653d-1ac0-4e25-b987-d0cb16dcacc9 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:09:06.432908 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 646/1279] 10.222.0.61 () {56 vars in 1109 bytes} [Mon Aug 22 19:09:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:09:06.438438 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50793d8e-2b3d-4c63-b1b9-34d7c795d646 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:09:06.468730 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:09:06.536275 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50793d8e-2b3d-4c63-b1b9-34d7c795d646 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:09:06.568680 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfea0b3f-4296-4bce-afc4-432b21c6175b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:09:06.569288 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfea0b3f-4296-4bce-afc4-432b21c6175b None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:09:06.569565 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfea0b3f-4296-4bce-afc4-432b21c6175b None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:09:06.569835 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfea0b3f-4296-4bce-afc4-432b21c6175b None admin] PATH_INFO: `/v3/users/placement` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:09:06.577476 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfea0b3f-4296-4bce-afc4-432b21c6175b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:09:06.578888 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfea0b3f-4296-4bce-afc4-432b21c6175b None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:09:06.590102 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-cfea0b3f-4296-4bce-afc4-432b21c6175b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 22 19:09:06.590102 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:09:06.590102 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:09:06.590102 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:09:06.590102 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:09:06.590102 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:09:06.590102 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:09:06.590102 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:09:06.590102 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:09:06.590102 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:09:06.590102 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:09:06.590102 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:09:06.590102 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 19:09:06.590102 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 19:09:06.590102 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 19:09:06.590102 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 19:09:06.591194 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:09:06.591194 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:09:06.591194 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 19:09:06.591194 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 19:09:06.591194 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 19:09:06.591194 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 19:09:06.591194 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:09:06.591194 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:09:06.591194 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:09:06.591194 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:09:06.591194 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:09:06.591194 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:09:06.591194 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:09:06.591194 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:09:06.591194 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:09:06.591194 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:09:06.591194 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:09:06.591194 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:09:06.592428 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:09:06.592428 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:09:06.592428 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 19:09:06.592428 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 19:09:06.592428 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 19:09:06.592428 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 19:09:06.592428 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 19:09:06.592428 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 19:09:06.592428 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 22 19:09:06.592428 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:09:06.592428 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 634/1280] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:09:06 2022] GET /identity/v3/users/placement => generated 87 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:09:06.596798 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91b653d-1ac0-4e25-b987-d0cb16dcacc9 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:09:06.600594 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:09:06.694938 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91b653d-1ac0-4e25-b987-d0cb16dcacc9 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:09:06.727021 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a5030a-6bd9-4500-b751-122c4332dfb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:09:06.728177 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a5030a-6bd9-4500-b751-122c4332dfb7 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:09:06.728454 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a5030a-6bd9-4500-b751-122c4332dfb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:09:06.728723 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a5030a-6bd9-4500-b751-122c4332dfb7 None admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:09:06.729439 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a5030a-6bd9-4500-b751-122c4332dfb7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:09:06.729742 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a5030a-6bd9-4500-b751-122c4332dfb7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:09:06.731154 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a5030a-6bd9-4500-b751-122c4332dfb7 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:09:06.748240 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 647/1281] 10.222.0.61 () {56 vars in 1143 bytes} [Mon Aug 22 19:09:06 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:09:06.753931 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfea0b3f-4296-4bce-afc4-432b21c6175b None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:09:06.843461 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:09:06.853367 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfea0b3f-4296-4bce-afc4-432b21c6175b None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:09:06.886016 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babdb415-9c06-426a-82fa-478844edc992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:09:06.886641 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babdb415-9c06-426a-82fa-478844edc992 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:09:06.886882 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babdb415-9c06-426a-82fa-478844edc992 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:09:06.887154 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babdb415-9c06-426a-82fa-478844edc992 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:09:06.952154 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babdb415-9c06-426a-82fa-478844edc992 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:09:06.953514 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babdb415-9c06-426a-82fa-478844edc992 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:09:07.038529 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:09:07.048979 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 635/1282] 10.222.0.61 () {58 vars in 1282 bytes} [Mon Aug 22 19:09:06 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 22 19:09:07.054707 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a5030a-6bd9-4500-b751-122c4332dfb7 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:09:07.114590 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:09:07.152674 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a5030a-6bd9-4500-b751-122c4332dfb7 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:09:07.185413 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b698f0-e4ef-42fe-9521-71b22e5a02ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:09:07.186029 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b698f0-e4ef-42fe-9521-71b22e5a02ef None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:09:07.186303 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b698f0-e4ef-42fe-9521-71b22e5a02ef None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:09:07.186572 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b698f0-e4ef-42fe-9521-71b22e5a02ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:09:07.196055 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b698f0-e4ef-42fe-9521-71b22e5a02ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:09:07.198100 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b698f0-e4ef-42fe-9521-71b22e5a02ef None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:09:07.212009 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-99b698f0-e4ef-42fe-9521-71b22e5a02ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:09:07.212009 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:09:07.212009 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:09:07.212009 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:09:07.212009 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:09:07.212009 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:09:07.212009 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:09:07.212009 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:09:07.212009 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:09:07.212009 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:09:07.212009 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:09:07.212009 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:09:07.212009 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 19:09:07.212009 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 19:09:07.212009 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 19:09:07.212009 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 19:09:07.213112 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:09:07.213112 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:09:07.213112 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:09:07.213112 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:09:07.213112 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:09:07.213112 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:09:07.213112 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:09:07.213112 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:09:07.213112 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:09:07.213112 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:09:07.213112 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:09:07.213112 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:09:07.213112 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:09:07.213112 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:09:07.213112 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:09:07.213112 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:09:07.213112 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 19:09:07.214383 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 19:09:07.214383 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 19:09:07.214383 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:09:07.214383 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 648/1283] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:09:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:09:07.218667 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babdb415-9c06-426a-82fa-478844edc992 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:09:07.313879 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babdb415-9c06-426a-82fa-478844edc992 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:09:07.346946 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e4bf46-5dfe-4067-a161-a8e775222233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:09:07.347537 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e4bf46-5dfe-4067-a161-a8e775222233 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:09:07.347845 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e4bf46-5dfe-4067-a161-a8e775222233 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:09:07.348115 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e4bf46-5dfe-4067-a161-a8e775222233 None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:09:07.348750 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e4bf46-5dfe-4067-a161-a8e775222233 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:09:07.349047 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e4bf46-5dfe-4067-a161-a8e775222233 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:09:07.350393 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e4bf46-5dfe-4067-a161-a8e775222233 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:09:07.361963 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 636/1284] 10.222.0.61 () {56 vars in 1109 bytes} [Mon Aug 22 19:09:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:09:07.367498 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b698f0-e4ef-42fe-9521-71b22e5a02ef None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:09:07.463541 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b698f0-e4ef-42fe-9521-71b22e5a02ef None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:09:07.476405 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:09:07.497115 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7173b2b-0d83-426d-b526-7c84ad216190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:09:07.497725 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7173b2b-0d83-426d-b526-7c84ad216190 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:09:07.497998 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7173b2b-0d83-426d-b526-7c84ad216190 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:09:07.498266 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7173b2b-0d83-426d-b526-7c84ad216190 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:09:07.504863 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7173b2b-0d83-426d-b526-7c84ad216190 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:09:07.506306 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7173b2b-0d83-426d-b526-7c84ad216190 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:09:07.518542 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-b7173b2b-0d83-426d-b526-7c84ad216190 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 19:09:07.518542 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:09:07.518542 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:09:07.518542 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:09:07.518542 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:09:07.518542 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:09:07.518542 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:09:07.518542 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:09:07.518542 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:09:07.518542 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:09:07.518542 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:09:07.518542 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:09:07.518542 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 19:09:07.518542 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 19:09:07.518542 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 19:09:07.519603 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 19:09:07.519603 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:09:07.519603 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:09:07.519603 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:09:07.519603 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:09:07.519603 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:09:07.519603 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:09:07.519603 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:09:07.519603 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 22 19:09:07.519603 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:09:07.519603 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:09:07.519603 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:09:07.519603 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:09:07.519603 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:09:07.519603 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:09:07.519603 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:09:07.519603 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:09:07.520972 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 19:09:07.520972 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 19:09:07.520972 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 19:09:07.520972 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 19:09:07.520972 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 19:09:07.520972 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 19:09:07.520972 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 19:09:07.520972 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 22 19:09:07.520972 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 649/1285] 10.222.0.61 () {56 vars in 1111 bytes} [Mon Aug 22 19:09:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:09:07.525340 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e4bf46-5dfe-4067-a161-a8e775222233 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:09:07.573938 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:09:07.621965 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e4bf46-5dfe-4067-a161-a8e775222233 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:09:07.653372 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca599de-ccc6-480a-b442-8100d920dd3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:09:07.654001 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca599de-ccc6-480a-b442-8100d920dd3d None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:09:07.654281 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca599de-ccc6-480a-b442-8100d920dd3d None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:09:07.654553 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca599de-ccc6-480a-b442-8100d920dd3d None admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:09:07.655262 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca599de-ccc6-480a-b442-8100d920dd3d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:09:07.655563 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca599de-ccc6-480a-b442-8100d920dd3d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:09:07.657010 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca599de-ccc6-480a-b442-8100d920dd3d None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:09:07.669160 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 637/1286] 10.222.0.61 () {56 vars in 1148 bytes} [Mon Aug 22 19:09:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:09:07.674961 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7173b2b-0d83-426d-b526-7c84ad216190 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:09:07.772976 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7173b2b-0d83-426d-b526-7c84ad216190 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:09:07.807004 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435b5ce3-9936-45fa-8eb5-f81dd1e0cd27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:09:07.807587 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435b5ce3-9936-45fa-8eb5-f81dd1e0cd27 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:09:07.807893 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435b5ce3-9936-45fa-8eb5-f81dd1e0cd27 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:09:07.808166 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435b5ce3-9936-45fa-8eb5-f81dd1e0cd27 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:09:07.808884 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435b5ce3-9936-45fa-8eb5-f81dd1e0cd27 None admin] RBAC: Adding query filter params (role.id=163c30c3b3c246beb849ce176f02d9da, scope.project.id=e07dc5ebdc504c6da706a2d21c0a3970, user.id=eef0f73297e04d62bc22acb790fad6ab) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:09:07.809191 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435b5ce3-9936-45fa-8eb5-f81dd1e0cd27 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:09:07.810600 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435b5ce3-9936-45fa-8eb5-f81dd1e0cd27 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:09:07.816272 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 650/1287] 10.222.0.61 () {56 vars in 1374 bytes} [Mon Aug 22 19:09:07 2022] GET /identity/v3/role_assignments?user.id=eef0f73297e04d62bc22acb790fad6ab&scope.project.id=e07dc5ebdc504c6da706a2d21c0a3970&role.id=163c30c3b3c246beb849ce176f02d9da => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:09:09.705761 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1021fe-cd94-43eb-8e73-bc9092aff5a0 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:09:09.706290 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1021fe-cd94-43eb-8e73-bc9092aff5a0 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:09:09.706290 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1021fe-cd94-43eb-8e73-bc9092aff5a0 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:09:09.707678 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 638/1288] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:09:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:09:09.713335 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f91d023-68f0-49cf-a72b-433cdf596fc3 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:09:09.713740 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f91d023-68f0-49cf-a72b-433cdf596fc3 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:09:09.713877 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f91d023-68f0-49cf-a72b-433cdf596fc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:09:09.731481 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-0f91d023-68f0-49cf-a72b-433cdf596fc3 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:09:09.732682 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-0f91d023-68f0-49cf-a72b-433cdf596fc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:09:09.761289 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:09:09.798168 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 651/1289] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:09:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 19:09:10.072888 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1021fe-cd94-43eb-8e73-bc9092aff5a0 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:09:10.171747 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1021fe-cd94-43eb-8e73-bc9092aff5a0 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:09:10.182475 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:09:10.205818 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e4ce2e-8fd1-4d76-971a-bec4cccabed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:09:10.206439 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e4ce2e-8fd1-4d76-971a-bec4cccabed4 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:09:10.206719 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e4ce2e-8fd1-4d76-971a-bec4cccabed4 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:09:10.206991 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e4ce2e-8fd1-4d76-971a-bec4cccabed4 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:09:10.207568 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e4ce2e-8fd1-4d76-971a-bec4cccabed4 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:09:10.208943 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e4ce2e-8fd1-4d76-971a-bec4cccabed4 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:09:10.217370 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-56e4ce2e-8fd1-4d76-971a-bec4cccabed4 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 22 19:09:10.217370 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:09:10.217370 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 19:09:10.217370 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 19:09:10.217370 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 19:09:10.217370 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 19:09:10.217370 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 19:09:10.217370 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 19:09:10.217370 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 22 19:09:10.217370 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:09:10.217370 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 19:09:10.217370 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:09:10.217370 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:09:10.217370 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:09:10.217370 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:09:10.217370 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:09:10.217370 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:09:10.218691 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:09:10.218691 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:09:10.218691 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:09:10.218691 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:09:10.218691 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:09:10.218691 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:09:10.218691 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 19:09:10.218691 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 19:09:10.218691 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 19:09:10.218691 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 19:09:10.218691 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:09:10.218691 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:09:10.218691 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:09:10.218691 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:09:10.218691 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:09:10.218691 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:09:10.219898 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:09:10.219898 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:09:10.219898 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:09:10.219898 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:09:10.219898 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:09:10.219898 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:09:10.219898 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:09:10.219898 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:09:10.219898 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:09:10.219898 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:09:10.219898 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 19:09:10.219898 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 19:09:10.219898 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 22 19:09:10.219898 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:09:10.219898 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 639/1290] 10.222.0.61 () {56 vars in 1117 bytes} [Mon Aug 22 19:09:10 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:09:10.224392 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f91d023-68f0-49cf-a72b-433cdf596fc3 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:09:10.322086 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f91d023-68f0-49cf-a72b-433cdf596fc3 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:09:10.355422 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dded01ad-bcc9-4d3b-8665-814549b93506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:09:10.355987 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dded01ad-bcc9-4d3b-8665-814549b93506 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:09:10.356267 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dded01ad-bcc9-4d3b-8665-814549b93506 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:09:10.356537 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dded01ad-bcc9-4d3b-8665-814549b93506 None admin] PATH_INFO: `/v3/services` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:09:10.357168 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dded01ad-bcc9-4d3b-8665-814549b93506 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:09:10.357468 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dded01ad-bcc9-4d3b-8665-814549b93506 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:09:10.358851 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dded01ad-bcc9-4d3b-8665-814549b93506 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:09:10.363787 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 652/1291] 10.222.0.61 () {56 vars in 1116 bytes} [Mon Aug 22 19:09:10 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:09:10.369979 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e4ce2e-8fd1-4d76-971a-bec4cccabed4 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:09:10.467669 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e4ce2e-8fd1-4d76-971a-bec4cccabed4 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:09:10.500971 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32639e39-df12-4116-ab94-d621e2538a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:09:10.501560 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32639e39-df12-4116-ab94-d621e2538a3f None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:09:10.501837 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32639e39-df12-4116-ab94-d621e2538a3f None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:09:10.502106 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32639e39-df12-4116-ab94-d621e2538a3f None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:09:10.502730 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32639e39-df12-4116-ab94-d621e2538a3f None admin] RBAC: Adding query filter params (type=placement) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:09:10.503027 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32639e39-df12-4116-ab94-d621e2538a3f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:09:10.504415 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32639e39-df12-4116-ab94-d621e2538a3f None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:09:10.509258 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 640/1292] 10.222.0.61 () {56 vars in 1116 bytes} [Mon Aug 22 19:09:10 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:09:12.371548 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49cf786-5e75-41d1-b189-e18a54847ffc None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:09:12.372154 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49cf786-5e75-41d1-b189-e18a54847ffc None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:09:12.372154 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49cf786-5e75-41d1-b189-e18a54847ffc None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:09:12.373467 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 653/1293] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:09:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:09:12.378539 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e685af-9368-4f85-8e86-d81490eb2121 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:09:12.379004 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e685af-9368-4f85-8e86-d81490eb2121 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:09:12.379109 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e685af-9368-4f85-8e86-d81490eb2121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:09:12.396608 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-83e685af-9368-4f85-8e86-d81490eb2121 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:09:12.397723 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-83e685af-9368-4f85-8e86-d81490eb2121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:09:12.427657 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:09:12.464123 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 641/1294] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:09:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 19:09:12.736083 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49cf786-5e75-41d1-b189-e18a54847ffc None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:09:12.766932 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:09:12.835733 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49cf786-5e75-41d1-b189-e18a54847ffc None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:09:12.868946 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710c7393-a021-4a7f-8d24-222f3e205f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:09:12.869533 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710c7393-a021-4a7f-8d24-222f3e205f73 None admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:09:12.869806 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710c7393-a021-4a7f-8d24-222f3e205f73 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:09:12.870073 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710c7393-a021-4a7f-8d24-222f3e205f73 None admin] PATH_INFO: `/v3/services` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:09:12.870698 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710c7393-a021-4a7f-8d24-222f3e205f73 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:09:12.872076 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710c7393-a021-4a7f-8d24-222f3e205f73 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:09:12.885449 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 654/1295] 10.222.0.61 () {60 vars in 1141 bytes} [Mon Aug 22 19:09:12 2022] POST /identity/v3/services => generated 246 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:09:14.744402 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b855f2b-1bd6-4e15-b7f6-4436332a7f19 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:09:14.744970 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b855f2b-1bd6-4e15-b7f6-4436332a7f19 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:09:14.744970 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b855f2b-1bd6-4e15-b7f6-4436332a7f19 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:09:14.746277 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 642/1296] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:09:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:09:14.751392 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c125c03-6b1d-4fc8-bbb0-92351d0acfca None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:09:14.751856 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c125c03-6b1d-4fc8-bbb0-92351d0acfca None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:09:14.751959 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c125c03-6b1d-4fc8-bbb0-92351d0acfca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:09:14.770156 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-3c125c03-6b1d-4fc8-bbb0-92351d0acfca None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:09:14.771339 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-3c125c03-6b1d-4fc8-bbb0-92351d0acfca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:09:14.840870 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 655/1297] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:09:14 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 19:09:15.107608 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b855f2b-1bd6-4e15-b7f6-4436332a7f19 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:09:15.209892 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b855f2b-1bd6-4e15-b7f6-4436332a7f19 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:09:15.242789 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27210a43-91f1-4f48-83e9-60290f75c00a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:09:15.243401 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27210a43-91f1-4f48-83e9-60290f75c00a None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:09:15.243695 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27210a43-91f1-4f48-83e9-60290f75c00a None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:09:15.243961 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27210a43-91f1-4f48-83e9-60290f75c00a None admin] PATH_INFO: `/v3/services/placement` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:09:15.244539 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27210a43-91f1-4f48-83e9-60290f75c00a None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:09:15.245882 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27210a43-91f1-4f48-83e9-60290f75c00a None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:09:15.254180 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-27210a43-91f1-4f48-83e9-60290f75c00a None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 22 19:09:15.254180 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:09:15.254180 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 19:09:15.254180 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 19:09:15.254180 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 19:09:15.254180 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 19:09:15.254180 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 19:09:15.254180 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 19:09:15.254180 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 22 19:09:15.254180 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:09:15.254180 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 19:09:15.254180 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:09:15.254180 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:09:15.254180 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:09:15.254180 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:09:15.254180 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:09:15.254180 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:09:15.255442 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:09:15.255442 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:09:15.255442 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:09:15.255442 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:09:15.255442 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:09:15.255442 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:09:15.255442 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 19:09:15.255442 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 19:09:15.255442 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 19:09:15.255442 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 19:09:15.255442 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:09:15.255442 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:09:15.255442 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:09:15.255442 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:09:15.255442 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:09:15.255442 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:09:15.256416 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:09:15.256416 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:09:15.256416 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:09:15.256416 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:09:15.256416 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:09:15.256416 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:09:15.256416 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:09:15.256416 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:09:15.256416 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:09:15.256416 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:09:15.256416 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 19:09:15.256416 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 19:09:15.256416 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 22 19:09:15.256416 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:09:15.256416 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 643/1298] 10.222.0.61 () {56 vars in 1117 bytes} [Mon Aug 22 19:09:15 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:09:15.261229 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c125c03-6b1d-4fc8-bbb0-92351d0acfca None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:09:15.279637 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:09:15.359437 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c125c03-6b1d-4fc8-bbb0-92351d0acfca None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:09:15.393593 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f20576-5468-4057-9af0-637242abce03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:09:15.394190 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f20576-5468-4057-9af0-637242abce03 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:09:15.394461 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f20576-5468-4057-9af0-637242abce03 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:09:15.394727 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f20576-5468-4057-9af0-637242abce03 None admin] PATH_INFO: `/v3/services` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:09:15.395355 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f20576-5468-4057-9af0-637242abce03 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:09:15.395672 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f20576-5468-4057-9af0-637242abce03 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:09:15.397033 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f20576-5468-4057-9af0-637242abce03 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:09:15.402480 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 656/1299] 10.222.0.61 () {56 vars in 1116 bytes} [Mon Aug 22 19:09:15 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:09:15.408289 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27210a43-91f1-4f48-83e9-60290f75c00a None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:09:15.438927 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:09:15.505219 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27210a43-91f1-4f48-83e9-60290f75c00a None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:09:15.539156 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9759e52f-dbee-4743-812a-f18839168933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:09:15.539807 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9759e52f-dbee-4743-812a-f18839168933 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:09:15.540084 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9759e52f-dbee-4743-812a-f18839168933 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:09:15.540351 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9759e52f-dbee-4743-812a-f18839168933 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:09:15.541050 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9759e52f-dbee-4743-812a-f18839168933 None admin] RBAC: Adding query filter params (interface=public, service_id=aaa3dc3dd8e74c9a9b65da9e17fae2f3, region_id=RegionOne) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:09:15.541354 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9759e52f-dbee-4743-812a-f18839168933 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:09:15.542697 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9759e52f-dbee-4743-812a-f18839168933 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:09:15.547496 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 644/1300] 10.222.0.61 () {56 vars in 1251 bytes} [Mon Aug 22 19:09:15 2022] GET /identity/v3/endpoints?service_id=aaa3dc3dd8e74c9a9b65da9e17fae2f3&interface=public®ion_id=RegionOne => generated 194 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:09:15.553056 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f20576-5468-4057-9af0-637242abce03 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:09:15.634292 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:09:15.652012 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f20576-5468-4057-9af0-637242abce03 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:09:15.685093 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f111597-bff8-4db1-8365-57428a90f88a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:09:15.685686 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f111597-bff8-4db1-8365-57428a90f88a None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:09:15.685962 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f111597-bff8-4db1-8365-57428a90f88a None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:09:15.686232 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f111597-bff8-4db1-8365-57428a90f88a None admin] PATH_INFO: `/v3/services` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:09:15.686832 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f111597-bff8-4db1-8365-57428a90f88a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:09:15.688220 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f111597-bff8-4db1-8365-57428a90f88a None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:09:15.694759 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 657/1301] 10.222.0.61 () {56 vars in 1087 bytes} [Mon Aug 22 19:09:15 2022] GET /identity/v3/services => generated 2191 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:09:17.577436 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee0d9f6-d803-47d2-afa8-0e078d9a5653 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:09:17.577997 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee0d9f6-d803-47d2-afa8-0e078d9a5653 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:09:17.577997 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee0d9f6-d803-47d2-afa8-0e078d9a5653 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:09:17.579305 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 645/1302] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:09:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:09:17.584380 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e178f6b5-22cf-4803-8f36-2925a7d1c484 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:09:17.584800 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e178f6b5-22cf-4803-8f36-2925a7d1c484 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:09:17.584951 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e178f6b5-22cf-4803-8f36-2925a7d1c484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:09:17.603574 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-e178f6b5-22cf-4803-8f36-2925a7d1c484 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:09:17.604789 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-e178f6b5-22cf-4803-8f36-2925a7d1c484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:09:17.669111 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 658/1303] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:09:17 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 19:09:17.933890 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee0d9f6-d803-47d2-afa8-0e078d9a5653 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:09:18.025712 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:09:18.032439 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee0d9f6-d803-47d2-afa8-0e078d9a5653 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:09:18.064444 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a99aec-2b10-48f7-acd8-1c558b1242e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:09:18.065057 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a99aec-2b10-48f7-acd8-1c558b1242e1 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:09:18.065330 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a99aec-2b10-48f7-acd8-1c558b1242e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:09:18.065597 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a99aec-2b10-48f7-acd8-1c558b1242e1 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:09:18.066168 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a99aec-2b10-48f7-acd8-1c558b1242e1 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:09:18.067502 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a99aec-2b10-48f7-acd8-1c558b1242e1 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:09:18.076410 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-08a99aec-2b10-48f7-acd8-1c558b1242e1 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 22 19:09:18.076410 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:09:18.076410 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 19:09:18.076410 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 19:09:18.076410 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 19:09:18.076410 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 19:09:18.076410 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 19:09:18.076410 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 19:09:18.076410 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 22 19:09:18.076410 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:09:18.076410 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 19:09:18.076410 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:09:18.076410 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 19:09:18.076410 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 19:09:18.076410 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 19:09:18.076410 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 19:09:18.076410 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 19:09:18.077599 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 19:09:18.077599 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 19:09:18.077599 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 19:09:18.077599 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 19:09:18.077599 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 19:09:18.077599 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 19:09:18.077599 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 19:09:18.077599 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 19:09:18.077599 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 19:09:18.077599 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 19:09:18.077599 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 19:09:18.077599 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 19:09:18.077599 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 19:09:18.077599 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 19:09:18.077599 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 19:09:18.077599 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 19:09:18.078661 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 19:09:18.078661 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 22 19:09:18.078661 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 19:09:18.078661 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 22 19:09:18.078661 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 19:09:18.078661 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 19:09:18.078661 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 19:09:18.078661 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 22 19:09:18.078661 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 19:09:18.078661 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 22 19:09:18.078661 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 19:09:18.078661 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 19:09:18.078661 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 22 19:09:18.078661 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 22 19:09:18.078661 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 646/1304] 10.222.0.61 () {56 vars in 1117 bytes} [Mon Aug 22 19:09:17 2022] GET /identity/v3/services/placement => generated 90 bytes in 144 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 19:09:18.083514 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e178f6b5-22cf-4803-8f36-2925a7d1c484 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:09:18.157945 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:09:18.180610 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e178f6b5-22cf-4803-8f36-2925a7d1c484 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:09:18.214346 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bfd9e1-6ce7-4940-9560-09c32fa6242c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:09:18.214889 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bfd9e1-6ce7-4940-9560-09c32fa6242c None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:09:18.215167 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bfd9e1-6ce7-4940-9560-09c32fa6242c None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:09:18.215438 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bfd9e1-6ce7-4940-9560-09c32fa6242c None admin] PATH_INFO: `/v3/services` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:09:18.216095 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bfd9e1-6ce7-4940-9560-09c32fa6242c None admin] RBAC: Adding query filter params (name=placement) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:09:18.216396 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bfd9e1-6ce7-4940-9560-09c32fa6242c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:09:18.217754 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bfd9e1-6ce7-4940-9560-09c32fa6242c None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:09:18.222712 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 659/1305] 10.222.0.61 () {56 vars in 1116 bytes} [Mon Aug 22 19:09:18 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:09:18.228619 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a99aec-2b10-48f7-acd8-1c558b1242e1 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:09:18.326817 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a99aec-2b10-48f7-acd8-1c558b1242e1 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:09:18.358930 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c854ea81-a39b-4900-9cbf-6e82f8a16a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:09:18.359519 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c854ea81-a39b-4900-9cbf-6e82f8a16a0d None admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:09:18.360146 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c854ea81-a39b-4900-9cbf-6e82f8a16a0d None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:09:18.360421 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c854ea81-a39b-4900-9cbf-6e82f8a16a0d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:09:18.361070 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c854ea81-a39b-4900-9cbf-6e82f8a16a0d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:09:18.362412 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c854ea81-a39b-4900-9cbf-6e82f8a16a0d None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:09:18.382608 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 647/1306] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 19:09:18 2022] POST /identity/v3/endpoints => generated 331 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:09:28.219893 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=30 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:09:28.378794 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:10:21.513538 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c73e81-a29e-46b3-b1db-b2840823af64 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:10:21.514470 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c73e81-a29e-46b3-b1db-b2840823af64 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:10:21.514897 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c73e81-a29e-46b3-b1db-b2840823af64 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:10:21.516445 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 660/1307] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:10:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:10:21.522652 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9beecd-2747-4746-88ba-c15084ad8c8f None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:10:21.523432 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9beecd-2747-4746-88ba-c15084ad8c8f None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:10:21.523888 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9beecd-2747-4746-88ba-c15084ad8c8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:10:21.603458 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-1d9beecd-2747-4746-88ba-c15084ad8c8f None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:10:21.604946 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-1d9beecd-2747-4746-88ba-c15084ad8c8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:10:21.844680 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 648/1308] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:10:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:10:22.139404 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be8ae63-babd-4160-b961-e12eee6908af None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:10:22.139988 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be8ae63-babd-4160-b961-e12eee6908af None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:10:22.140410 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be8ae63-babd-4160-b961-e12eee6908af None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:10:22.142341 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 661/1309] 10.222.0.61 () {54 vars in 965 bytes} [Mon Aug 22 19:10:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:10:22.150273 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf32a7f-4eee-405c-b161-96bd3c61275e None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:10:22.150565 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf32a7f-4eee-405c-b161-96bd3c61275e None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:10:22.150847 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf32a7f-4eee-405c-b161-96bd3c61275e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:10:22.236526 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-adf32a7f-4eee-405c-b161-96bd3c61275e None None] MFA Rules not processed for user `a4aca06c966a42f29f5c5bc108529d41`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:10:22.237829 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-adf32a7f-4eee-405c-b161-96bd3c61275e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:10:22.261552 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:10:22.324676 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 649/1310] 10.222.0.61 () {58 vars in 1062 bytes} [Mon Aug 22 19:10:22 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:10:22.334511 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be8ae63-babd-4160-b961-e12eee6908af None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:10:22.336371 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-3be8ae63-babd-4160-b961-e12eee6908af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:10:22.456195 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be8ae63-babd-4160-b961-e12eee6908af None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:10:22.535496 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5720f488-3d53-461f-8156-7bc2bfaac953 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:10:22.536197 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5720f488-3d53-461f-8156-7bc2bfaac953 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:10:22.536949 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5720f488-3d53-461f-8156-7bc2bfaac953 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:10:22.537144 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5720f488-3d53-461f-8156-7bc2bfaac953 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:10:22.538864 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-5720f488-3d53-461f-8156-7bc2bfaac953 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:10:22.596272 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:10:22.655835 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5720f488-3d53-461f-8156-7bc2bfaac953 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:10:22.657459 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5720f488-3d53-461f-8156-7bc2bfaac953 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:10:22.678693 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 662/1311] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:10:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:10:24.893317 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05501e76-b6a2-4ada-99de-cfab7715bf77 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:10:24.893779 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05501e76-b6a2-4ada-99de-cfab7715bf77 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:10:24.893977 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05501e76-b6a2-4ada-99de-cfab7715bf77 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:10:24.895429 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 650/1312] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:10:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:10:24.900860 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d1a1fa-1eef-425d-b507-e682c2133f6b None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:10:24.901282 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d1a1fa-1eef-425d-b507-e682c2133f6b None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:10:24.901470 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d1a1fa-1eef-425d-b507-e682c2133f6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:10:24.983576 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-37d1a1fa-1eef-425d-b507-e682c2133f6b None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:10:24.984828 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-37d1a1fa-1eef-425d-b507-e682c2133f6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:10:25.079145 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 663/1313] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:10:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:10:25.360368 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05501e76-b6a2-4ada-99de-cfab7715bf77 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:10:25.361937 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-05501e76-b6a2-4ada-99de-cfab7715bf77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:10:25.463465 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05501e76-b6a2-4ada-99de-cfab7715bf77 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:10:25.534233 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:10:25.536920 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489d2f35-4732-4f27-94ed-23b8bea75b92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:10:25.537566 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489d2f35-4732-4f27-94ed-23b8bea75b92 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:10:25.538106 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489d2f35-4732-4f27-94ed-23b8bea75b92 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:10:25.538610 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489d2f35-4732-4f27-94ed-23b8bea75b92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:10:25.540500 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-489d2f35-4732-4f27-94ed-23b8bea75b92 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:10:25.647996 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489d2f35-4732-4f27-94ed-23b8bea75b92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:10:25.649405 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489d2f35-4732-4f27-94ed-23b8bea75b92 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:10:25.666837 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 651/1314] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:10:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:10:27.907727 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd5589b-6f49-4f8b-afbc-b55330fb3d6c None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:10:27.908313 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd5589b-6f49-4f8b-afbc-b55330fb3d6c None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:10:27.908313 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd5589b-6f49-4f8b-afbc-b55330fb3d6c None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:10:27.909636 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 664/1315] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:10:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:10:27.915535 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4016dc0-6824-47f9-85d0-28e35403c73d None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:10:27.916026 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4016dc0-6824-47f9-85d0-28e35403c73d None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:10:27.916149 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4016dc0-6824-47f9-85d0-28e35403c73d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:10:27.980594 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-d4016dc0-6824-47f9-85d0-28e35403c73d None None] MFA Rules not processed for user `a0683ebbf01f453fb5544a458adb02f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:10:27.981814 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-d4016dc0-6824-47f9-85d0-28e35403c73d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:10:28.032312 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:10:28.076507 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 652/1316] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:10:27 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:10:28.356805 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd5589b-6f49-4f8b-afbc-b55330fb3d6c None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:10:28.358603 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-8fd5589b-6f49-4f8b-afbc-b55330fb3d6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:10:28.368950 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:10:28.459538 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd5589b-6f49-4f8b-afbc-b55330fb3d6c None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:10:28.529982 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b11158-4475-44e3-8dac-5f4af04dd193 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:10:28.530648 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b11158-4475-44e3-8dac-5f4af04dd193 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:10:28.530857 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b11158-4475-44e3-8dac-5f4af04dd193 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:10:28.531123 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b11158-4475-44e3-8dac-5f4af04dd193 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:10:28.532763 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-13b11158-4475-44e3-8dac-5f4af04dd193 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:10:28.594149 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:10:28.640744 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b11158-4475-44e3-8dac-5f4af04dd193 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:10:28.642085 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b11158-4475-44e3-8dac-5f4af04dd193 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:10:28.659447 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 665/1317] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:10:28 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:10:31.141808 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6ff257-dcfd-4cbd-8b7a-e6797720f99a None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:10:31.142339 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6ff257-dcfd-4cbd-8b7a-e6797720f99a None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:10:31.142339 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6ff257-dcfd-4cbd-8b7a-e6797720f99a None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:10:31.143728 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 653/1318] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:10:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:10:31.149519 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6628117-3cef-4696-b79b-eca148486880 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:10:31.149900 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6628117-3cef-4696-b79b-eca148486880 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:10:31.150021 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6628117-3cef-4696-b79b-eca148486880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:10:31.214135 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-c6628117-3cef-4696-b79b-eca148486880 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:10:31.215342 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-c6628117-3cef-4696-b79b-eca148486880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:10:31.309423 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 666/1319] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:10:31 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 19:10:31.598043 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6ff257-dcfd-4cbd-8b7a-e6797720f99a None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:10:31.599660 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-7a6ff257-dcfd-4cbd-8b7a-e6797720f99a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:10:31.702612 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6ff257-dcfd-4cbd-8b7a-e6797720f99a None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:10:31.777622 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2487e417-cf63-4e39-b1b1-bcb2d3017b0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:10:31.778221 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2487e417-cf63-4e39-b1b1-bcb2d3017b0c service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:10:31.778500 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2487e417-cf63-4e39-b1b1-bcb2d3017b0c service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:10:31.778770 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2487e417-cf63-4e39-b1b1-bcb2d3017b0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:10:31.780408 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-2487e417-cf63-4e39-b1b1-bcb2d3017b0c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:10:31.787735 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:10:31.902020 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2487e417-cf63-4e39-b1b1-bcb2d3017b0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:10:31.903360 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2487e417-cf63-4e39-b1b1-bcb2d3017b0c service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:10:31.921296 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 654/1320] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:10:31 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:10:34.459397 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4005ee9-c1b1-474e-8cc2-1bb07b319a94 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:10:34.460015 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4005ee9-c1b1-474e-8cc2-1bb07b319a94 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:10:34.460015 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4005ee9-c1b1-474e-8cc2-1bb07b319a94 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:10:34.461315 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 667/1321] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:10:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:10:34.466502 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72a9c9a-a2b0-402c-85c4-20a2f2b39af2 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:10:34.466931 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72a9c9a-a2b0-402c-85c4-20a2f2b39af2 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:10:34.467044 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72a9c9a-a2b0-402c-85c4-20a2f2b39af2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:10:34.533690 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-c72a9c9a-a2b0-402c-85c4-20a2f2b39af2 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:10:34.535407 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-c72a9c9a-a2b0-402c-85c4-20a2f2b39af2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:10:34.592146 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:10:34.631380 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 655/1322] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:10:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:10:34.912070 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4005ee9-c1b1-474e-8cc2-1bb07b319a94 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:10:34.913773 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-c4005ee9-c1b1-474e-8cc2-1bb07b319a94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:10:34.933475 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:10:35.021917 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4005ee9-c1b1-474e-8cc2-1bb07b319a94 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:10:35.102396 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59f7f93-6631-4b31-a3a9-4acc2e7e4f14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:10:35.103003 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59f7f93-6631-4b31-a3a9-4acc2e7e4f14 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:10:35.103282 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59f7f93-6631-4b31-a3a9-4acc2e7e4f14 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:10:35.103555 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59f7f93-6631-4b31-a3a9-4acc2e7e4f14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:10:35.105184 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-a59f7f93-6631-4b31-a3a9-4acc2e7e4f14 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:10:35.182590 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:10:35.220586 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59f7f93-6631-4b31-a3a9-4acc2e7e4f14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:10:35.222444 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59f7f93-6631-4b31-a3a9-4acc2e7e4f14 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:10:35.245088 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 668/1323] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:10:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:10:37.232489 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f212d41-3aff-4020-aed3-f31191e3e554 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:10:37.233011 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f212d41-3aff-4020-aed3-f31191e3e554 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:10:37.233011 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f212d41-3aff-4020-aed3-f31191e3e554 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:10:37.234457 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 656/1324] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:10:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:10:37.240279 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97411a09-a9aa-476f-b546-fe895e5dcc93 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:10:37.240748 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97411a09-a9aa-476f-b546-fe895e5dcc93 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:10:37.240842 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97411a09-a9aa-476f-b546-fe895e5dcc93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:10:37.306798 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-97411a09-a9aa-476f-b546-fe895e5dcc93 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:10:37.308038 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-97411a09-a9aa-476f-b546-fe895e5dcc93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:10:37.405037 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 669/1325] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:10:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:10:37.692911 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f212d41-3aff-4020-aed3-f31191e3e554 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:10:37.694444 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-5f212d41-3aff-4020-aed3-f31191e3e554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:10:37.800543 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f212d41-3aff-4020-aed3-f31191e3e554 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:10:37.883768 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02ad331-c7fc-4a26-8b05-7e9f7c626c00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:10:37.884373 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02ad331-c7fc-4a26-8b05-7e9f7c626c00 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:10:37.884648 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02ad331-c7fc-4a26-8b05-7e9f7c626c00 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:10:37.884917 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02ad331-c7fc-4a26-8b05-7e9f7c626c00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:10:37.886519 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-c02ad331-c7fc-4a26-8b05-7e9f7c626c00 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:10:37.903405 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:10:37.995260 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02ad331-c7fc-4a26-8b05-7e9f7c626c00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:10:37.996724 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02ad331-c7fc-4a26-8b05-7e9f7c626c00 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:10:38.014986 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 657/1326] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:10:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:10:39.961566 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0e7a54-0e85-4fa7-b5b4-2ad2e80bec98 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:10:39.962045 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0e7a54-0e85-4fa7-b5b4-2ad2e80bec98 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:10:39.962138 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0e7a54-0e85-4fa7-b5b4-2ad2e80bec98 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:10:39.963492 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 670/1327] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:10:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:10:39.968890 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6556a9fa-5515-419d-8e75-e71f34073a50 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:10:39.969310 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6556a9fa-5515-419d-8e75-e71f34073a50 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:10:39.969436 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6556a9fa-5515-419d-8e75-e71f34073a50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:10:40.041640 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-6556a9fa-5515-419d-8e75-e71f34073a50 None None] MFA Rules not processed for user `a0683ebbf01f453fb5544a458adb02f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:10:40.042865 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-6556a9fa-5515-419d-8e75-e71f34073a50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:10:40.107719 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:10:40.141756 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 658/1328] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:10:39 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:10:40.421682 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0e7a54-0e85-4fa7-b5b4-2ad2e80bec98 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:10:40.423228 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-1e0e7a54-0e85-4fa7-b5b4-2ad2e80bec98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:10:40.461801 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:10:40.540469 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0e7a54-0e85-4fa7-b5b4-2ad2e80bec98 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:10:40.612542 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82160d12-bdfd-470b-824b-cdafc51bc973 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:10:40.613152 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82160d12-bdfd-470b-824b-cdafc51bc973 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:10:40.613475 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82160d12-bdfd-470b-824b-cdafc51bc973 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:10:40.614352 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82160d12-bdfd-470b-824b-cdafc51bc973 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:10:40.615970 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-82160d12-bdfd-470b-824b-cdafc51bc973 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:10:40.691841 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:10:40.724927 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82160d12-bdfd-470b-824b-cdafc51bc973 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:10:40.726359 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82160d12-bdfd-470b-824b-cdafc51bc973 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:10:40.745053 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 671/1329] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:10:40 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:10:42.920447 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2974008a-1036-4df9-b17a-9c17b98f5463 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:10:42.920905 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2974008a-1036-4df9-b17a-9c17b98f5463 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:10:42.920997 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2974008a-1036-4df9-b17a-9c17b98f5463 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:10:42.922358 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 659/1330] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:10:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:10:42.928442 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e622f9f1-6a25-4998-868c-51d7b12b202a None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:10:42.928933 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e622f9f1-6a25-4998-868c-51d7b12b202a None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:10:42.929118 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e622f9f1-6a25-4998-868c-51d7b12b202a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:10:42.998879 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-e622f9f1-6a25-4998-868c-51d7b12b202a None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:10:43.000154 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-e622f9f1-6a25-4998-868c-51d7b12b202a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:10:43.098784 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 672/1331] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:10:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:10:43.379889 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2974008a-1036-4df9-b17a-9c17b98f5463 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:10:43.381433 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-2974008a-1036-4df9-b17a-9c17b98f5463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:10:43.482197 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2974008a-1036-4df9-b17a-9c17b98f5463 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:10:43.555817 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d32b1d6-342a-42a2-9437-ab29d65808f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:10:43.556426 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d32b1d6-342a-42a2-9437-ab29d65808f1 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:10:43.556705 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d32b1d6-342a-42a2-9437-ab29d65808f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:10:43.556975 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d32b1d6-342a-42a2-9437-ab29d65808f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:10:43.558619 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-6d32b1d6-342a-42a2-9437-ab29d65808f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:10:43.585781 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:10:43.666714 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d32b1d6-342a-42a2-9437-ab29d65808f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:10:43.668103 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d32b1d6-342a-42a2-9437-ab29d65808f1 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:10:43.686020 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 660/1332] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:10:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:10:45.622321 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68618998-6249-4188-9bde-2332b0679d6f None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:10:45.622930 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68618998-6249-4188-9bde-2332b0679d6f None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:10:45.622930 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68618998-6249-4188-9bde-2332b0679d6f None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:10:45.624260 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 673/1333] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:10:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:10:45.629317 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d64822c-0db1-4c06-970c-e7e06084124e None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:10:45.629792 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d64822c-0db1-4c06-970c-e7e06084124e None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:10:45.629902 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d64822c-0db1-4c06-970c-e7e06084124e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:10:45.694980 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-3d64822c-0db1-4c06-970c-e7e06084124e None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:10:45.696222 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-3d64822c-0db1-4c06-970c-e7e06084124e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:10:45.769781 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:10:45.792242 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 661/1334] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:10:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:10:46.075122 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68618998-6249-4188-9bde-2332b0679d6f None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:10:46.076718 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-68618998-6249-4188-9bde-2332b0679d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:10:46.117415 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:10:46.183511 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68618998-6249-4188-9bde-2332b0679d6f None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:10:46.255304 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7636b72-3684-418c-a648-e6d0eaa0fdb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:10:46.255933 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7636b72-3684-418c-a648-e6d0eaa0fdb3 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:10:46.256212 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7636b72-3684-418c-a648-e6d0eaa0fdb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:10:46.256480 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7636b72-3684-418c-a648-e6d0eaa0fdb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:10:46.258083 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-a7636b72-3684-418c-a648-e6d0eaa0fdb3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:10:46.338722 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:10:46.365138 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7636b72-3684-418c-a648-e6d0eaa0fdb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:10:46.366561 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7636b72-3684-418c-a648-e6d0eaa0fdb3 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:10:46.384260 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 674/1335] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:10:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:10:48.598723 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ba3b04-7eba-4551-b02e-88db8740c6b6 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:10:48.599299 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ba3b04-7eba-4551-b02e-88db8740c6b6 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:10:48.599299 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ba3b04-7eba-4551-b02e-88db8740c6b6 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:10:48.600637 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 662/1336] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:10:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:10:48.605921 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1128de0e-921d-4572-8c22-44f34cc285de None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:10:48.606363 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1128de0e-921d-4572-8c22-44f34cc285de None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:10:48.606729 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1128de0e-921d-4572-8c22-44f34cc285de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:10:48.671652 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-1128de0e-921d-4572-8c22-44f34cc285de None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:10:48.672873 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-1128de0e-921d-4572-8c22-44f34cc285de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:10:48.770154 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 675/1337] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:10:48 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:10:49.064139 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07042b2-3b12-4446-8c29-ec68bf9bb99c None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:10:49.064420 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07042b2-3b12-4446-8c29-ec68bf9bb99c None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:10:49.064693 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07042b2-3b12-4446-8c29-ec68bf9bb99c None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:10:49.066093 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 663/1338] 10.222.0.61 () {54 vars in 965 bytes} [Mon Aug 22 19:10:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:10:49.072381 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f00879-7d16-416e-a767-3d2171480a9e None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:10:49.072668 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f00879-7d16-416e-a767-3d2171480a9e None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:10:49.072942 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f00879-7d16-416e-a767-3d2171480a9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:10:49.126165 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:10:49.148805 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-f6f00879-7d16-416e-a767-3d2171480a9e None None] MFA Rules not processed for user `a4aca06c966a42f29f5c5bc108529d41`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:10:49.150017 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-f6f00879-7d16-416e-a767-3d2171480a9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:10:49.223417 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 676/1339] 10.222.0.61 () {58 vars in 1062 bytes} [Mon Aug 22 19:10:49 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:10:49.232509 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07042b2-3b12-4446-8c29-ec68bf9bb99c None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:10:49.234038 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-c07042b2-3b12-4446-8c29-ec68bf9bb99c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:10:49.334765 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07042b2-3b12-4446-8c29-ec68bf9bb99c None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:10:49.405055 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d80190-dd98-494b-afaa-6327afca04b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:10:49.405645 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d80190-dd98-494b-afaa-6327afca04b9 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:10:49.405919 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d80190-dd98-494b-afaa-6327afca04b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:10:49.406185 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d80190-dd98-494b-afaa-6327afca04b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:10:49.407773 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-11d80190-dd98-494b-afaa-6327afca04b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:10:49.443170 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:10:49.516050 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d80190-dd98-494b-afaa-6327afca04b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:10:49.517387 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d80190-dd98-494b-afaa-6327afca04b9 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:10:49.537117 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 664/1340] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:10:49 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:10:53.878232 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19df6fc5-6807-4299-8ea5-b4da1865f5bf None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:10:53.879379 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19df6fc5-6807-4299-8ea5-b4da1865f5bf None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:10:53.879379 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19df6fc5-6807-4299-8ea5-b4da1865f5bf None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:10:53.880746 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 677/1341] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:10:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:10:53.885959 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfc3197-8ef2-4bd0-aba4-deb67486f922 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:10:53.886397 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfc3197-8ef2-4bd0-aba4-deb67486f922 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:10:53.886511 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfc3197-8ef2-4bd0-aba4-deb67486f922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:10:53.954667 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-8dfc3197-8ef2-4bd0-aba4-deb67486f922 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:10:53.955911 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-8dfc3197-8ef2-4bd0-aba4-deb67486f922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:10:54.044368 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:10:54.056378 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 665/1342] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:10:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:10:54.341017 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19df6fc5-6807-4299-8ea5-b4da1865f5bf None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:10:54.342571 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-19df6fc5-6807-4299-8ea5-b4da1865f5bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:10:54.448439 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19df6fc5-6807-4299-8ea5-b4da1865f5bf None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:10:54.487885 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:10:54.523059 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f25098-542d-4f6d-9f28-f4da222b3061 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:10:54.523682 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f25098-542d-4f6d-9f28-f4da222b3061 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:10:54.523961 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f25098-542d-4f6d-9f28-f4da222b3061 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:10:54.524227 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f25098-542d-4f6d-9f28-f4da222b3061 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:10:54.525891 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-37f25098-542d-4f6d-9f28-f4da222b3061 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:10:54.634736 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f25098-542d-4f6d-9f28-f4da222b3061 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:10:54.636124 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f25098-542d-4f6d-9f28-f4da222b3061 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:10:54.654471 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 678/1343] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:10:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:10:57.224551 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ebcc4e-444d-43f8-9044-7c378c69d80f None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:10:57.225128 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ebcc4e-444d-43f8-9044-7c378c69d80f None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:10:57.225226 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ebcc4e-444d-43f8-9044-7c378c69d80f None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:10:57.226712 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 666/1344] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:10:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:10:57.232202 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dad0a3-ef68-4807-a8ed-29147e859cc4 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:10:57.232640 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dad0a3-ef68-4807-a8ed-29147e859cc4 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:10:57.232754 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dad0a3-ef68-4807-a8ed-29147e859cc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:10:57.299385 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-20dad0a3-ef68-4807-a8ed-29147e859cc4 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:10:57.301437 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-20dad0a3-ef68-4807-a8ed-29147e859cc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:10:57.319911 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:10:57.401175 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 679/1345] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:10:57 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:10:57.689090 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ebcc4e-444d-43f8-9044-7c378c69d80f None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:10:57.691259 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-e6ebcc4e-444d-43f8-9044-7c378c69d80f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:10:57.797311 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ebcc4e-444d-43f8-9044-7c378c69d80f None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:10:57.875215 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec5f3dd-ab4c-41b4-b31c-8f8f874fe896 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:10:57.875879 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec5f3dd-ab4c-41b4-b31c-8f8f874fe896 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:10:57.876767 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec5f3dd-ab4c-41b4-b31c-8f8f874fe896 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:10:57.877043 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec5f3dd-ab4c-41b4-b31c-8f8f874fe896 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:10:57.879198 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-1ec5f3dd-ab4c-41b4-b31c-8f8f874fe896 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:10:57.923846 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:10:57.992824 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec5f3dd-ab4c-41b4-b31c-8f8f874fe896 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:10:57.994174 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec5f3dd-ab4c-41b4-b31c-8f8f874fe896 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:10:58.014366 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 667/1346] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:10:57 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:11:01.843469 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baba217-dcf6-4ff6-8f66-2a35ca057177 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:11:01.843983 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baba217-dcf6-4ff6-8f66-2a35ca057177 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:11:01.844078 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baba217-dcf6-4ff6-8f66-2a35ca057177 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:11:01.845413 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 680/1347] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:11:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:11:01.851115 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0781144-4e2a-45ff-804a-2b86160b83d1 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:11:01.851549 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0781144-4e2a-45ff-804a-2b86160b83d1 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:11:01.851673 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0781144-4e2a-45ff-804a-2b86160b83d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:11:01.925647 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-f0781144-4e2a-45ff-804a-2b86160b83d1 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:11:01.928164 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-f0781144-4e2a-45ff-804a-2b86160b83d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:11:02.028325 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:11:02.035586 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 668/1348] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:11:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:11:02.318932 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baba217-dcf6-4ff6-8f66-2a35ca057177 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:11:02.320512 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-0baba217-dcf6-4ff6-8f66-2a35ca057177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:11:02.428269 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baba217-dcf6-4ff6-8f66-2a35ca057177 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:11:02.477962 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:11:02.506245 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e794d7-99a6-431c-89bb-b3c75f49d918 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:11:02.506892 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e794d7-99a6-431c-89bb-b3c75f49d918 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:11:02.507175 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e794d7-99a6-431c-89bb-b3c75f49d918 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:11:02.507447 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e794d7-99a6-431c-89bb-b3c75f49d918 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:11:02.509129 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-32e794d7-99a6-431c-89bb-b3c75f49d918 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:11:02.617909 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e794d7-99a6-431c-89bb-b3c75f49d918 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:11:02.619263 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e794d7-99a6-431c-89bb-b3c75f49d918 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:11:02.637651 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 681/1349] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:11:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:11:07.494988 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69d1ceb-f9c3-4a73-9974-287625502a19 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:11:07.495514 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69d1ceb-f9c3-4a73-9974-287625502a19 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:11:07.495514 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69d1ceb-f9c3-4a73-9974-287625502a19 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:11:07.497281 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 669/1350] 10.222.0.61 () {54 vars in 921 bytes} [Mon Aug 22 19:11:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:11:07.502587 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bd4aca-09d3-4a6c-974f-7b2d0aeed750 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:11:07.503069 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bd4aca-09d3-4a6c-974f-7b2d0aeed750 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:11:07.503250 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bd4aca-09d3-4a6c-974f-7b2d0aeed750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:11:07.589378 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-c5bd4aca-09d3-4a6c-974f-7b2d0aeed750 None None] MFA Rules not processed for user `eef0f73297e04d62bc22acb790fad6ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:11:07.590826 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-c5bd4aca-09d3-4a6c-974f-7b2d0aeed750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:11:07.610614 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:11:07.761783 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 682/1351] 10.222.0.61 () {58 vars in 1018 bytes} [Mon Aug 22 19:11:07 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:11:11.963741 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02619512-62a6-4604-8d9c-b447fdde108b None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:11:11.964272 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02619512-62a6-4604-8d9c-b447fdde108b None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:11:11.964272 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02619512-62a6-4604-8d9c-b447fdde108b None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:11:11.965821 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 670/1352] 10.222.0.61 () {54 vars in 921 bytes} [Mon Aug 22 19:11:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:11:11.970605 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f0955a-3def-4729-9d5d-350879291ba8 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:11:11.971089 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f0955a-3def-4729-9d5d-350879291ba8 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:11:11.971350 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f0955a-3def-4729-9d5d-350879291ba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:11:12.050920 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-89f0955a-3def-4729-9d5d-350879291ba8 None None] MFA Rules not processed for user `eef0f73297e04d62bc22acb790fad6ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:11:12.052824 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-89f0955a-3def-4729-9d5d-350879291ba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:11:12.130748 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:11:12.170682 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 683/1353] 10.222.0.61 () {58 vars in 1018 bytes} [Mon Aug 22 19:11:11 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:11:13.507472 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc3569a-f1cf-4923-a182-68864cfab5ac None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:11:13.508047 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc3569a-f1cf-4923-a182-68864cfab5ac None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:11:13.508175 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc3569a-f1cf-4923-a182-68864cfab5ac None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:11:13.509530 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 671/1354] 10.222.0.61 () {54 vars in 921 bytes} [Mon Aug 22 19:11:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:11:13.514563 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3943e4-dcae-4d19-a2cd-7ec640ef6bd6 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:11:13.515071 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3943e4-dcae-4d19-a2cd-7ec640ef6bd6 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:11:13.515160 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3943e4-dcae-4d19-a2cd-7ec640ef6bd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:11:13.591705 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-ff3943e4-dcae-4d19-a2cd-7ec640ef6bd6 None None] MFA Rules not processed for user `eef0f73297e04d62bc22acb790fad6ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:11:13.592950 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-ff3943e4-dcae-4d19-a2cd-7ec640ef6bd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:11:13.700339 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 684/1355] 10.222.0.61 () {58 vars in 1018 bytes} [Mon Aug 22 19:11:13 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 19:11:16.979967 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e882fbc7-3dac-4d3a-80b0-d09225498e37 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:11:16.981368 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e882fbc7-3dac-4d3a-80b0-d09225498e37 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:11:16.981834 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e882fbc7-3dac-4d3a-80b0-d09225498e37 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:11:16.983396 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 672/1356] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:11:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:11:16.989123 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3885a43-2f3d-4029-bedc-eee61b83f389 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:11:16.989633 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3885a43-2f3d-4029-bedc-eee61b83f389 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:11:16.989739 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3885a43-2f3d-4029-bedc-eee61b83f389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:11:17.038786 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:11:17.078830 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-d3885a43-2f3d-4029-bedc-eee61b83f389 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:11:17.081629 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-d3885a43-2f3d-4029-bedc-eee61b83f389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:11:17.209659 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 685/1357] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:11:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:11:17.512733 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89f78c1-ea51-48bd-9a32-60e6959d901d None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:11:17.514145 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89f78c1-ea51-48bd-9a32-60e6959d901d None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:11:17.514969 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89f78c1-ea51-48bd-9a32-60e6959d901d None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:11:17.516859 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 673/1358] 10.222.0.61 () {54 vars in 962 bytes} [Mon Aug 22 19:11:17 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:11:17.526143 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c783f0-7afb-4772-b3ca-698f2b18918c None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:11:17.526731 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c783f0-7afb-4772-b3ca-698f2b18918c None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:11:17.527226 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c783f0-7afb-4772-b3ca-698f2b18918c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:11:17.607869 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-51c783f0-7afb-4772-b3ca-698f2b18918c None None] MFA Rules not processed for user `e7f43e2756b746a6825ca1a06b7d7c91`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:11:17.609182 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-51c783f0-7afb-4772-b3ca-698f2b18918c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:11:17.659923 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:11:17.742825 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 686/1359] 10.222.0.61 () {58 vars in 1059 bytes} [Mon Aug 22 19:11:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:11:17.750823 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89f78c1-ea51-48bd-9a32-60e6959d901d None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:11:17.752492 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-a89f78c1-ea51-48bd-9a32-60e6959d901d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:11:17.906773 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89f78c1-ea51-48bd-9a32-60e6959d901d None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:11:17.993005 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:11:18.035510 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119229e1-a4de-4642-82d7-3c9682e59940 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f43e2756b746a6825ca1a06b7d7c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:11:18.036160 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119229e1-a4de-4642-82d7-3c9682e59940 service nova] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:11:18.036442 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119229e1-a4de-4642-82d7-3c9682e59940 service nova] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:11:18.036714 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119229e1-a4de-4642-82d7-3c9682e59940 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:11:18.038353 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-119229e1-a4de-4642-82d7-3c9682e59940 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:11:18.130436 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119229e1-a4de-4642-82d7-3c9682e59940 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:11:18.132060 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119229e1-a4de-4642-82d7-3c9682e59940 service nova] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:11:18.151082 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 674/1360] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:11:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 19:11:20.267102 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634e5a9a-b299-4239-93f5-488a0046abea None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:11:20.267680 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634e5a9a-b299-4239-93f5-488a0046abea None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:11:20.267680 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634e5a9a-b299-4239-93f5-488a0046abea None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:11:20.269168 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 687/1361] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:11:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:11:20.274346 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8567f19-da07-4bab-928c-0d5d04851df5 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:11:20.274788 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8567f19-da07-4bab-928c-0d5d04851df5 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:11:20.274899 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8567f19-da07-4bab-928c-0d5d04851df5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:11:20.344877 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-a8567f19-da07-4bab-928c-0d5d04851df5 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:11:20.346725 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-a8567f19-da07-4bab-928c-0d5d04851df5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:11:20.389434 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:11:20.459118 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 675/1362] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:11:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:11:20.755518 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634e5a9a-b299-4239-93f5-488a0046abea None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:11:20.757073 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-634e5a9a-b299-4239-93f5-488a0046abea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:11:20.915073 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634e5a9a-b299-4239-93f5-488a0046abea None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:11:20.926067 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:11:21.037429 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babc2075-16c5-4d32-b243-1934dfd11612 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f43e2756b746a6825ca1a06b7d7c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:11:21.038414 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babc2075-16c5-4d32-b243-1934dfd11612 service nova] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:11:21.038758 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babc2075-16c5-4d32-b243-1934dfd11612 service nova] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:11:21.039107 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babc2075-16c5-4d32-b243-1934dfd11612 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:11:21.041075 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-babc2075-16c5-4d32-b243-1934dfd11612 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:11:21.151313 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babc2075-16c5-4d32-b243-1934dfd11612 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:11:21.152971 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babc2075-16c5-4d32-b243-1934dfd11612 service nova] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:11:21.172202 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 688/1363] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:11:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:11:23.409180 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6936710-a951-4e3a-bbd2-5699331abf7f None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:11:23.409769 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6936710-a951-4e3a-bbd2-5699331abf7f None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:11:23.409769 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6936710-a951-4e3a-bbd2-5699331abf7f None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:11:23.411244 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 676/1364] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:11:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:11:23.416451 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9983dc5e-c30d-42d5-85ed-736ebed335ef None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:11:23.416867 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9983dc5e-c30d-42d5-85ed-736ebed335ef None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:11:23.416990 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9983dc5e-c30d-42d5-85ed-736ebed335ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:11:23.433696 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:11:23.490230 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-9983dc5e-c30d-42d5-85ed-736ebed335ef None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:11:23.491365 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-9983dc5e-c30d-42d5-85ed-736ebed335ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:11:23.594079 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 689/1365] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:11:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:11:23.887292 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd40b9db-29db-4b17-abc5-f7c48794c700 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:11:23.887754 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd40b9db-29db-4b17-abc5-f7c48794c700 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:11:23.888249 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd40b9db-29db-4b17-abc5-f7c48794c700 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:11:23.889841 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 677/1366] 10.222.0.61 () {54 vars in 962 bytes} [Mon Aug 22 19:11:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:11:23.896040 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050374dc-d3bb-4a6a-b9c9-5d13b38528dc None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:11:23.896329 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050374dc-d3bb-4a6a-b9c9-5d13b38528dc None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:11:23.896606 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050374dc-d3bb-4a6a-b9c9-5d13b38528dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:11:23.976020 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-050374dc-d3bb-4a6a-b9c9-5d13b38528dc None None] MFA Rules not processed for user `e7f43e2756b746a6825ca1a06b7d7c91`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:11:23.977377 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-050374dc-d3bb-4a6a-b9c9-5d13b38528dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:11:24.014220 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:11:24.103966 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 690/1367] 10.222.0.61 () {58 vars in 1059 bytes} [Mon Aug 22 19:11:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:11:24.117175 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd40b9db-29db-4b17-abc5-f7c48794c700 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:11:24.118718 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-cd40b9db-29db-4b17-abc5-f7c48794c700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:11:24.278400 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd40b9db-29db-4b17-abc5-f7c48794c700 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:11:24.301942 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:11:24.401924 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fe0199-36e5-4647-a60e-e8d537010eab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f43e2756b746a6825ca1a06b7d7c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:11:24.402437 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fe0199-36e5-4647-a60e-e8d537010eab service nova] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:11:24.402712 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fe0199-36e5-4647-a60e-e8d537010eab service nova] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:11:24.402978 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fe0199-36e5-4647-a60e-e8d537010eab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:11:24.404602 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-87fe0199-36e5-4647-a60e-e8d537010eab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:11:24.496165 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fe0199-36e5-4647-a60e-e8d537010eab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:11:24.497761 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fe0199-36e5-4647-a60e-e8d537010eab service nova] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:11:24.519134 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 678/1368] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:11:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 19:11:25.136178 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb22e312-846d-436a-98e4-ccbbb6bf1651 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:11:25.136759 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb22e312-846d-436a-98e4-ccbbb6bf1651 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:11:25.136759 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb22e312-846d-436a-98e4-ccbbb6bf1651 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:11:25.138226 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 691/1369] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:11:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:11:25.143524 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e4a32c-8b0f-4074-a135-1692c7f0f454 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:11:25.143841 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e4a32c-8b0f-4074-a135-1692c7f0f454 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:11:25.144122 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e4a32c-8b0f-4074-a135-1692c7f0f454 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:11:25.176997 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:11:25.216963 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-77e4a32c-8b0f-4074-a135-1692c7f0f454 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:11:25.218317 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-77e4a32c-8b0f-4074-a135-1692c7f0f454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:11:25.327988 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 679/1370] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:11:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:11:25.910751 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcd6051-e96e-4ade-ac46-c2820d2dc16d None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:11:25.911158 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcd6051-e96e-4ade-ac46-c2820d2dc16d None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:11:25.911528 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcd6051-e96e-4ade-ac46-c2820d2dc16d None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:11:25.913140 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 692/1371] 10.222.0.61 () {54 vars in 964 bytes} [Mon Aug 22 19:11:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:11:25.918720 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4880833-70bd-4f77-a622-c166d8c64841 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:11:25.919198 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4880833-70bd-4f77-a622-c166d8c64841 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:11:25.919282 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4880833-70bd-4f77-a622-c166d8c64841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:11:26.002611 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-d4880833-70bd-4f77-a622-c166d8c64841 None None] MFA Rules not processed for user `56e79498873d4ed29a0e19bbea9b9214`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:11:26.003865 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-d4880833-70bd-4f77-a622-c166d8c64841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:11:26.054446 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:11:26.087239 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 680/1372] 10.222.0.61 () {58 vars in 1061 bytes} [Mon Aug 22 19:11:25 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 19:11:26.095359 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcd6051-e96e-4ade-ac46-c2820d2dc16d None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:11:26.097971 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-abcd6051-e96e-4ade-ac46-c2820d2dc16d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:11:26.216774 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcd6051-e96e-4ade-ac46-c2820d2dc16d None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:11:26.265026 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:11:26.310613 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee1cb87-c8d9-41f4-850a-bd150f134eb3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e79498873d4ed29a0e19bbea9b9214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:11:26.311422 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee1cb87-c8d9-41f4-850a-bd150f134eb3 service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:11:26.311839 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee1cb87-c8d9-41f4-850a-bd150f134eb3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:11:26.312211 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee1cb87-c8d9-41f4-850a-bd150f134eb3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:11:26.314063 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-aee1cb87-c8d9-41f4-850a-bd150f134eb3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:11:26.457448 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee1cb87-c8d9-41f4-850a-bd150f134eb3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:11:26.459488 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee1cb87-c8d9-41f4-850a-bd150f134eb3 service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:11:26.484580 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a4eb05-4810-4cbb-8e4b-c8341eee62cf None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:11:26.485056 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a4eb05-4810-4cbb-8e4b-c8341eee62cf None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:11:26.485209 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a4eb05-4810-4cbb-8e4b-c8341eee62cf None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:11:26.486954 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 681/1373] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:11:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:11:26.492626 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03ff9af-1068-4365-bd40-738e50e07593 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:11:26.492907 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03ff9af-1068-4365-bd40-738e50e07593 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:11:26.493273 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03ff9af-1068-4365-bd40-738e50e07593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:11:26.495270 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 693/1374] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:11:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:11:26.581770 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-d03ff9af-1068-4365-bd40-738e50e07593 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:11:26.583142 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-d03ff9af-1068-4365-bd40-738e50e07593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:11:26.687920 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 682/1375] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:11:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:11:26.975935 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee1cb87-c8d9-41f4-850a-bd150f134eb3 service cinder] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:11:26.977371 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-aee1cb87-c8d9-41f4-850a-bd150f134eb3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:11:27.042893 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:11:27.075291 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee1cb87-c8d9-41f4-850a-bd150f134eb3 service cinder] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:11:27.194084 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a871d64e-3b3b-43e0-baa4-5d3f8dfb137d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f43e2756b746a6825ca1a06b7d7c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:11:27.194719 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a871d64e-3b3b-43e0-baa4-5d3f8dfb137d service nova] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:11:27.195160 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a871d64e-3b3b-43e0-baa4-5d3f8dfb137d service nova] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:11:27.195604 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a871d64e-3b3b-43e0-baa4-5d3f8dfb137d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:11:27.197406 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-a871d64e-3b3b-43e0-baa4-5d3f8dfb137d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:11:27.277506 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:11:27.288513 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a871d64e-3b3b-43e0-baa4-5d3f8dfb137d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:11:27.290218 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a871d64e-3b3b-43e0-baa4-5d3f8dfb137d service nova] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:11:27.308358 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 694/1376] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:11:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:11:29.252216 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5b8118-c65b-4605-9b6e-0f9432a4c6b5 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:11:29.252737 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5b8118-c65b-4605-9b6e-0f9432a4c6b5 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:11:29.252737 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5b8118-c65b-4605-9b6e-0f9432a4c6b5 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:11:29.254146 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 683/1377] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:11:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:11:29.259443 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b09301-9049-4d36-8e8e-ed93ae3956df None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:11:29.259876 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b09301-9049-4d36-8e8e-ed93ae3956df None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:11:29.260028 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b09301-9049-4d36-8e8e-ed93ae3956df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:11:29.324804 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-91b09301-9049-4d36-8e8e-ed93ae3956df None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:11:29.326051 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-91b09301-9049-4d36-8e8e-ed93ae3956df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:11:29.429827 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 695/1378] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:11:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:11:29.721802 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5b8118-c65b-4605-9b6e-0f9432a4c6b5 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:11:29.723336 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-3d5b8118-c65b-4605-9b6e-0f9432a4c6b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:11:29.776754 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:11:29.879727 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5b8118-c65b-4605-9b6e-0f9432a4c6b5 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:11:30.035177 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3cad9d-5c25-477c-b992-dcd39668f7a4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f43e2756b746a6825ca1a06b7d7c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:11:30.035713 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3cad9d-5c25-477c-b992-dcd39668f7a4 service nova] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:11:30.035997 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3cad9d-5c25-477c-b992-dcd39668f7a4 service nova] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:11:30.036396 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3cad9d-5c25-477c-b992-dcd39668f7a4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:11:30.039073 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-fa3cad9d-5c25-477c-b992-dcd39668f7a4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:11:30.057531 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:11:30.161139 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3cad9d-5c25-477c-b992-dcd39668f7a4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:11:30.162922 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3cad9d-5c25-477c-b992-dcd39668f7a4 service nova] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:11:30.186977 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 684/1379] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:11:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:11:32.137816 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d0d18-f6b8-42ff-bbeb-9cc90aab96dc None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:11:32.138434 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d0d18-f6b8-42ff-bbeb-9cc90aab96dc None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:11:32.138434 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d0d18-f6b8-42ff-bbeb-9cc90aab96dc None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:11:32.147670 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa22d9e-9fd6-4072-a830-e3a5865d01ef None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:11:32.147670 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa22d9e-9fd6-4072-a830-e3a5865d01ef None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:11:32.147670 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa22d9e-9fd6-4072-a830-e3a5865d01ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:11:32.148329 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 696/1380] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:11:32 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:11:32.221492 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-baa22d9e-9fd6-4072-a830-e3a5865d01ef None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:11:32.222718 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-baa22d9e-9fd6-4072-a830-e3a5865d01ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:11:32.312263 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:11:32.326524 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 685/1381] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:11:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 19:11:32.616457 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377d0d18-f6b8-42ff-bbeb-9cc90aab96dc None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:11:32.617982 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-377d0d18-f6b8-42ff-bbeb-9cc90aab96dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:11:32.682542 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:11:32.772950 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377d0d18-f6b8-42ff-bbeb-9cc90aab96dc None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:11:32.908743 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24200f1-6a63-4aae-b803-e77880e1da70 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f43e2756b746a6825ca1a06b7d7c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:11:32.909274 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24200f1-6a63-4aae-b803-e77880e1da70 service nova] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:11:32.909554 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24200f1-6a63-4aae-b803-e77880e1da70 service nova] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:11:32.909832 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24200f1-6a63-4aae-b803-e77880e1da70 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:11:32.911436 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-f24200f1-6a63-4aae-b803-e77880e1da70 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:11:32.941515 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:11:33.012414 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24200f1-6a63-4aae-b803-e77880e1da70 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:11:33.014037 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24200f1-6a63-4aae-b803-e77880e1da70 service nova] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:11:33.034085 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 697/1382] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:11:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:11:35.024076 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb8ec84-7d4b-42ca-9eeb-2739460dba4e None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:11:35.024660 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb8ec84-7d4b-42ca-9eeb-2739460dba4e None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:11:35.024660 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb8ec84-7d4b-42ca-9eeb-2739460dba4e None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:11:35.026015 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 686/1383] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:11:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:11:35.032274 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4093f9b-2aa7-42f3-86a3-e73ad85fabf0 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:11:35.032737 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4093f9b-2aa7-42f3-86a3-e73ad85fabf0 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:11:35.032978 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4093f9b-2aa7-42f3-86a3-e73ad85fabf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:11:35.107366 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-f4093f9b-2aa7-42f3-86a3-e73ad85fabf0 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:11:35.108696 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-f4093f9b-2aa7-42f3-86a3-e73ad85fabf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:11:35.205879 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:11:35.209813 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 698/1384] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:11:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:11:35.493399 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb8ec84-7d4b-42ca-9eeb-2739460dba4e None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:11:35.494941 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-6eb8ec84-7d4b-42ca-9eeb-2739460dba4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:11:35.643465 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb8ec84-7d4b-42ca-9eeb-2739460dba4e None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:11:35.708466 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:11:35.761175 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3f8dcc-bf7d-4eff-b58d-4416632cf90c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f43e2756b746a6825ca1a06b7d7c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:11:35.761921 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3f8dcc-bf7d-4eff-b58d-4416632cf90c service nova] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:11:35.762281 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3f8dcc-bf7d-4eff-b58d-4416632cf90c service nova] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:11:35.762638 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3f8dcc-bf7d-4eff-b58d-4416632cf90c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:11:35.764743 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-5a3f8dcc-bf7d-4eff-b58d-4416632cf90c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:11:35.878746 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3f8dcc-bf7d-4eff-b58d-4416632cf90c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:11:35.880874 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3f8dcc-bf7d-4eff-b58d-4416632cf90c service nova] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:11:35.904362 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 687/1385] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:11:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:11:37.896652 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f3862c-fa90-4ec0-b19e-62657f6467cf None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:11:37.897256 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f3862c-fa90-4ec0-b19e-62657f6467cf None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:11:37.897256 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f3862c-fa90-4ec0-b19e-62657f6467cf None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:11:37.898653 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 699/1386] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:11:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:11:37.904535 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf40d26-d191-4cf1-b016-719e96c212b2 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:11:37.905003 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf40d26-d191-4cf1-b016-719e96c212b2 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:11:37.905196 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf40d26-d191-4cf1-b016-719e96c212b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:11:37.991208 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-ddf40d26-d191-4cf1-b016-719e96c212b2 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:11:37.992518 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-ddf40d26-d191-4cf1-b016-719e96c212b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:11:38.024593 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:11:38.119629 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 688/1387] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:11:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:11:38.505841 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f3862c-fa90-4ec0-b19e-62657f6467cf None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:11:38.507893 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-b9f3862c-fa90-4ec0-b19e-62657f6467cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:11:38.670746 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f3862c-fa90-4ec0-b19e-62657f6467cf None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:11:38.741699 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:11:38.791372 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083911ce-8f8d-435a-8f1b-c253dac01c60 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f43e2756b746a6825ca1a06b7d7c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:11:38.791901 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083911ce-8f8d-435a-8f1b-c253dac01c60 service nova] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:11:38.792187 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083911ce-8f8d-435a-8f1b-c253dac01c60 service nova] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:11:38.792462 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083911ce-8f8d-435a-8f1b-c253dac01c60 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:11:38.794081 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-083911ce-8f8d-435a-8f1b-c253dac01c60 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:11:38.892685 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083911ce-8f8d-435a-8f1b-c253dac01c60 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:11:38.894307 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083911ce-8f8d-435a-8f1b-c253dac01c60 service nova] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:11:38.912663 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 700/1388] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:11:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 19:11:40.828026 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaf6454-5fe0-495e-98a9-06ac2240f98c None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:11:40.828630 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaf6454-5fe0-495e-98a9-06ac2240f98c None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:11:40.828630 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaf6454-5fe0-495e-98a9-06ac2240f98c None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:11:40.830464 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 689/1389] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:11:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:11:40.835775 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f3b62e-ea56-49c7-93a2-7ee96a4c961a None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:11:40.836214 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f3b62e-ea56-49c7-93a2-7ee96a4c961a None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:11:40.836322 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f3b62e-ea56-49c7-93a2-7ee96a4c961a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:11:40.904305 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-72f3b62e-ea56-49c7-93a2-7ee96a4c961a None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:11:40.905513 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-72f3b62e-ea56-49c7-93a2-7ee96a4c961a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:11:40.939883 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:11:41.003038 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 701/1390] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:11:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:11:41.290414 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaf6454-5fe0-495e-98a9-06ac2240f98c None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:11:41.291975 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-ceaf6454-5fe0-495e-98a9-06ac2240f98c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:11:41.439293 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaf6454-5fe0-495e-98a9-06ac2240f98c None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:11:41.448457 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:11:41.553818 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5041f18e-aaf3-4588-9aa6-417ab92fc8dd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f43e2756b746a6825ca1a06b7d7c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:11:41.554434 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5041f18e-aaf3-4588-9aa6-417ab92fc8dd service nova] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:11:41.554705 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5041f18e-aaf3-4588-9aa6-417ab92fc8dd service nova] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:11:41.554968 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5041f18e-aaf3-4588-9aa6-417ab92fc8dd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:11:41.556593 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-5041f18e-aaf3-4588-9aa6-417ab92fc8dd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:11:41.647948 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5041f18e-aaf3-4588-9aa6-417ab92fc8dd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:11:41.649542 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5041f18e-aaf3-4588-9aa6-417ab92fc8dd service nova] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:11:41.668053 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 690/1391] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:11:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:11:41.888954 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6927570-2585-4912-8159-8758be8c5bd0 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:11:41.889493 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6927570-2585-4912-8159-8758be8c5bd0 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:11:41.889493 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6927570-2585-4912-8159-8758be8c5bd0 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:11:41.890927 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 702/1392] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:11:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:11:41.896351 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c2e69-4cda-452a-8497-dbd8ef3271a3 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:11:41.896823 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c2e69-4cda-452a-8497-dbd8ef3271a3 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:11:41.896923 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c2e69-4cda-452a-8497-dbd8ef3271a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:11:41.917250 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:11:41.970630 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-3e9c2e69-4cda-452a-8497-dbd8ef3271a3 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:11:41.971797 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-3e9c2e69-4cda-452a-8497-dbd8ef3271a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:11:42.070600 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 691/1393] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:11:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:11:42.371230 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5d2232-6c85-41b3-91ae-6d1004603545 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:11:42.371512 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5d2232-6c85-41b3-91ae-6d1004603545 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:11:42.371820 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5d2232-6c85-41b3-91ae-6d1004603545 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:11:42.373190 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 703/1394] 10.222.0.61 () {54 vars in 967 bytes} [Mon Aug 22 19:11:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 19:11:42.378281 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6944772b-554a-45a5-a57c-f427158e95a4 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:11:42.378569 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6944772b-554a-45a5-a57c-f427158e95a4 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:11:42.378843 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6944772b-554a-45a5-a57c-f427158e95a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:11:42.456423 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-6944772b-554a-45a5-a57c-f427158e95a4 None None] MFA Rules not processed for user `fe15d6a79ed44e5396d13cd29ba160ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:11:42.457644 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-6944772b-554a-45a5-a57c-f427158e95a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:11:42.493488 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:11:42.538014 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 692/1395] 10.222.0.61 () {58 vars in 1064 bytes} [Mon Aug 22 19:11:42 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:11:42.545821 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5d2232-6c85-41b3-91ae-6d1004603545 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:11:42.547319 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-8a5d2232-6c85-41b3-91ae-6d1004603545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:11:42.654439 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5d2232-6c85-41b3-91ae-6d1004603545 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:11:42.717491 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:11:42.730695 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e5396f-aad4-4fec-9dd7-781e7c16b266 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d6a79ed44e5396d13cd29ba160ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:11:42.731299 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e5396f-aad4-4fec-9dd7-781e7c16b266 service glance] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:11:42.731575 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e5396f-aad4-4fec-9dd7-781e7c16b266 service glance] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:11:42.731876 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e5396f-aad4-4fec-9dd7-781e7c16b266 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:11:42.733478 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-e1e5396f-aad4-4fec-9dd7-781e7c16b266 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:11:42.847547 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e5396f-aad4-4fec-9dd7-781e7c16b266 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:11:42.849011 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e5396f-aad4-4fec-9dd7-781e7c16b266 service glance] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:11:42.867768 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 704/1396] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:11:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:11:43.609648 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7cb4b5-8015-46b4-8618-e138bd23921c None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:11:43.610131 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7cb4b5-8015-46b4-8618-e138bd23921c None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:11:43.610239 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7cb4b5-8015-46b4-8618-e138bd23921c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:11:43.610755 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca5f25b-e2cb-4b23-8d5e-2550e5e8ada2 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:11:43.611140 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca5f25b-e2cb-4b23-8d5e-2550e5e8ada2 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:11:43.611373 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca5f25b-e2cb-4b23-8d5e-2550e5e8ada2 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:11:43.612945 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 705/1397] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:11:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 19:11:43.617972 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41b5e7b-a776-42d8-9e1b-caee65e406f3 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:11:43.618288 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41b5e7b-a776-42d8-9e1b-caee65e406f3 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:11:43.618548 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41b5e7b-a776-42d8-9e1b-caee65e406f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:11:43.678763 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-ab7cb4b5-8015-46b4-8618-e138bd23921c None None] MFA Rules not processed for user `399396a8fe254da199bc48b0ca5243d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:11:43.680230 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-ab7cb4b5-8015-46b4-8618-e138bd23921c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:11:43.687056 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-b41b5e7b-a776-42d8-9e1b-caee65e406f3 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:11:43.688813 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-b41b5e7b-a776-42d8-9e1b-caee65e406f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:11:43.732090 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:11:43.769491 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 693/1398] 10.222.0.61 () {58 vars in 1009 bytes} [Mon Aug 22 19:11:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 19:11:43.790133 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 706/1399] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:11:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 19:11:43.816595 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6d6df7-4b12-47fc-a784-dfb17dc4bff9 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:11:43.816949 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6d6df7-4b12-47fc-a784-dfb17dc4bff9 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:11:43.817288 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6d6df7-4b12-47fc-a784-dfb17dc4bff9 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:11:43.818761 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 694/1400] 10.222.0.61 () {54 vars in 943 bytes} [Mon Aug 22 19:11:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:11:43.824884 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10afd64-0750-447d-a55f-ffe7247b424c None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:11:43.825175 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10afd64-0750-447d-a55f-ffe7247b424c None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:11:43.825454 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10afd64-0750-447d-a55f-ffe7247b424c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:11:43.900649 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-b10afd64-0750-447d-a55f-ffe7247b424c None None] MFA Rules not processed for user `2824359491934d0dacaa21a718becee5`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:11:43.901786 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-b10afd64-0750-447d-a55f-ffe7247b424c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:11:44.011930 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 707/1401] 10.222.0.61 () {58 vars in 1040 bytes} [Mon Aug 22 19:11:43 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:11:44.021027 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6d6df7-4b12-47fc-a784-dfb17dc4bff9 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:11:44.022609 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-1d6d6df7-4b12-47fc-a784-dfb17dc4bff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:11:44.080386 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:11:44.090431 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10afd64-0750-447d-a55f-ffe7247b424c None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:11:44.091663 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-b10afd64-0750-447d-a55f-ffe7247b424c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:11:44.096286 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:11:44.173150 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6d6df7-4b12-47fc-a784-dfb17dc4bff9 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:11:44.197383 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10afd64-0750-447d-a55f-ffe7247b424c None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:11:44.278887 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ed54d2-ee56-406d-8eec-615740f9dbdb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2824359491934d0dacaa21a718becee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:11:44.279496 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ed54d2-ee56-406d-8eec-615740f9dbdb service swift] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:11:44.279816 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ed54d2-ee56-406d-8eec-615740f9dbdb service swift] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:11:44.280089 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ed54d2-ee56-406d-8eec-615740f9dbdb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:11:44.281743 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-96ed54d2-ee56-406d-8eec-615740f9dbdb service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:11:44.313702 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9616b132-9261-4637-b326-7c79c8317ee2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f43e2756b746a6825ca1a06b7d7c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:11:44.314317 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9616b132-9261-4637-b326-7c79c8317ee2 service nova] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:11:44.314596 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9616b132-9261-4637-b326-7c79c8317ee2 service nova] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:11:44.314867 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9616b132-9261-4637-b326-7c79c8317ee2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:11:44.315908 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:11:44.316853 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-9616b132-9261-4637-b326-7c79c8317ee2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:11:44.334419 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:11:44.368141 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ed54d2-ee56-406d-8eec-615740f9dbdb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:11:44.369727 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ed54d2-ee56-406d-8eec-615740f9dbdb service swift] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:11:44.378423 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 695/1402] 10.222.0.61 () {60 vars in 1384 bytes} [Mon Aug 22 19:11:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 22 19:11:44.413093 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9616b132-9261-4637-b326-7c79c8317ee2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:11:44.414750 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9616b132-9261-4637-b326-7c79c8317ee2 service nova] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:11:44.434778 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 708/1403] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:11:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 19:11:46.398600 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52d0e66-4191-416b-ade2-1dfbcd51a516 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:11:46.399173 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52d0e66-4191-416b-ade2-1dfbcd51a516 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:11:46.399173 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52d0e66-4191-416b-ade2-1dfbcd51a516 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:11:46.404106 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 696/1404] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:11:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:11:46.405578 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d352bb63-db90-4c0b-986e-508165b7f697 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:11:46.406011 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d352bb63-db90-4c0b-986e-508165b7f697 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:11:46.406102 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d352bb63-db90-4c0b-986e-508165b7f697 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:11:46.480269 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-d352bb63-db90-4c0b-986e-508165b7f697 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:11:46.480925 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-d352bb63-db90-4c0b-986e-508165b7f697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:11:46.583145 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:11:46.595539 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 709/1405] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:11:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 19:11:46.884747 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52d0e66-4191-416b-ade2-1dfbcd51a516 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:11:46.886365 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-b52d0e66-4191-416b-ade2-1dfbcd51a516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:11:47.052833 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52d0e66-4191-416b-ade2-1dfbcd51a516 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:11:47.084773 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:11:47.186260 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a79a22e-abae-482a-b7e5-075337bdf63d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f43e2756b746a6825ca1a06b7d7c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:11:47.186869 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a79a22e-abae-482a-b7e5-075337bdf63d service nova] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:11:47.187150 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a79a22e-abae-482a-b7e5-075337bdf63d service nova] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:11:47.187425 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a79a22e-abae-482a-b7e5-075337bdf63d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:11:47.189618 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-1a79a22e-abae-482a-b7e5-075337bdf63d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:11:47.295392 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a79a22e-abae-482a-b7e5-075337bdf63d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:11:47.297053 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a79a22e-abae-482a-b7e5-075337bdf63d service nova] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:11:47.317191 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 697/1406] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:11:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:11:49.272775 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25b2e05-1bdc-4c90-9230-c38c8634b162 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:11:49.273315 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25b2e05-1bdc-4c90-9230-c38c8634b162 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:11:49.273315 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25b2e05-1bdc-4c90-9230-c38c8634b162 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:11:49.274569 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 710/1407] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:11:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:11:49.279772 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026d7408-6a02-4a0f-8a49-525b55f709f2 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:11:49.280182 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026d7408-6a02-4a0f-8a49-525b55f709f2 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:11:49.280315 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026d7408-6a02-4a0f-8a49-525b55f709f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:11:49.332292 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:11:49.357547 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-026d7408-6a02-4a0f-8a49-525b55f709f2 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:11:49.358770 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-026d7408-6a02-4a0f-8a49-525b55f709f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:11:49.464442 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 698/1408] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:11:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:11:49.761160 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25b2e05-1bdc-4c90-9230-c38c8634b162 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:11:49.764322 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-a25b2e05-1bdc-4c90-9230-c38c8634b162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:11:49.931348 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25b2e05-1bdc-4c90-9230-c38c8634b162 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:11:50.013668 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:11:50.070840 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfe3456-b45f-40f2-8b2a-2f3bca8a2950 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f43e2756b746a6825ca1a06b7d7c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:11:50.071458 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfe3456-b45f-40f2-8b2a-2f3bca8a2950 service nova] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:11:50.071782 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfe3456-b45f-40f2-8b2a-2f3bca8a2950 service nova] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:11:50.072042 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfe3456-b45f-40f2-8b2a-2f3bca8a2950 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:11:50.074363 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-8cfe3456-b45f-40f2-8b2a-2f3bca8a2950 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:11:50.178125 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfe3456-b45f-40f2-8b2a-2f3bca8a2950 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:11:50.179748 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfe3456-b45f-40f2-8b2a-2f3bca8a2950 service nova] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:11:50.199232 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 711/1409] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:11:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:11:59.462784 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=50 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:12:00.196911 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=65 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:12:06.422551 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27664aac-6aa4-4361-8bb0-2b5dd7474460 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:12:06.423089 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27664aac-6aa4-4361-8bb0-2b5dd7474460 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:12:06.423089 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27664aac-6aa4-4361-8bb0-2b5dd7474460 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:12:06.424401 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 699/1410] 10.222.0.61 () {54 vars in 918 bytes} [Mon Aug 22 19:12:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:12:06.429647 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64bd5c1-2dc0-4477-9b47-572637148788 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:12:06.430057 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64bd5c1-2dc0-4477-9b47-572637148788 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:12:06.430268 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64bd5c1-2dc0-4477-9b47-572637148788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:12:06.506818 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-c64bd5c1-2dc0-4477-9b47-572637148788 None None] MFA Rules not processed for user `eef0f73297e04d62bc22acb790fad6ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:12:06.508062 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-c64bd5c1-2dc0-4477-9b47-572637148788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:12:06.611020 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 712/1411] 10.222.0.61 () {58 vars in 1015 bytes} [Mon Aug 22 19:12:06 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:12:06.625472 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217e3153-8675-408c-8ddc-43f83154fda7 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:12:06.625754 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217e3153-8675-408c-8ddc-43f83154fda7 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:12:06.626055 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217e3153-8675-408c-8ddc-43f83154fda7 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:12:06.627371 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 700/1412] 10.222.0.61 () {54 vars in 918 bytes} [Mon Aug 22 19:12:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:12:06.631904 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aa4727-bd4b-4386-8b5f-01ffb3e2c0eb None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:12:06.632221 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aa4727-bd4b-4386-8b5f-01ffb3e2c0eb None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:12:06.632494 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aa4727-bd4b-4386-8b5f-01ffb3e2c0eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:12:06.703488 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:12:06.706608 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-59aa4727-bd4b-4386-8b5f-01ffb3e2c0eb None None] MFA Rules not processed for user `e7f43e2756b746a6825ca1a06b7d7c91`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:12:06.709045 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-59aa4727-bd4b-4386-8b5f-01ffb3e2c0eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:12:06.826377 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 713/1413] 10.222.0.61 () {58 vars in 1015 bytes} [Mon Aug 22 19:12:06 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:12:09.186180 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4211612d-5c22-4c67-98b4-280257f0a1f7 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:12:09.186680 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4211612d-5c22-4c67-98b4-280257f0a1f7 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:12:09.187396 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4211612d-5c22-4c67-98b4-280257f0a1f7 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:12:09.188693 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 701/1414] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 22 19:12:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:12:09.193996 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db68f014-159f-4450-896b-db762bb7c2f3 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:12:09.194412 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db68f014-159f-4450-896b-db762bb7c2f3 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:12:09.194533 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db68f014-159f-4450-896b-db762bb7c2f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:12:09.259360 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-db68f014-159f-4450-896b-db762bb7c2f3 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:12:09.260556 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-db68f014-159f-4450-896b-db762bb7c2f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:12:09.315893 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:12:09.357239 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 714/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 22 19:12:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:12:09.641306 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4211612d-5c22-4c67-98b4-280257f0a1f7 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:12:09.642925 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-4211612d-5c22-4c67-98b4-280257f0a1f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:12:09.787106 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4211612d-5c22-4c67-98b4-280257f0a1f7 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:12:09.859607 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:12:09.899135 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e103abfa-3f5e-4ffb-ae70-b5ee22e549eb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f43e2756b746a6825ca1a06b7d7c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:12:09.899695 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e103abfa-3f5e-4ffb-ae70-b5ee22e549eb service nova] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:12:09.899977 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e103abfa-3f5e-4ffb-ae70-b5ee22e549eb service nova] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:12:09.900246 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e103abfa-3f5e-4ffb-ae70-b5ee22e549eb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:12:09.901854 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-e103abfa-3f5e-4ffb-ae70-b5ee22e549eb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:12:09.990633 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e103abfa-3f5e-4ffb-ae70-b5ee22e549eb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:12:09.992235 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e103abfa-3f5e-4ffb-ae70-b5ee22e549eb service nova] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:12:10.009699 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 702/1416] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:12:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:12:10.893029 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e300d23-4395-4002-bd45-06f5b8e560f7 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:12:10.893609 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e300d23-4395-4002-bd45-06f5b8e560f7 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:12:10.893609 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e300d23-4395-4002-bd45-06f5b8e560f7 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:12:10.894946 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 715/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 22 19:12:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:12:10.899782 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133c57cb-1e04-4fea-9178-c30ee8065495 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:12:10.900171 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133c57cb-1e04-4fea-9178-c30ee8065495 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:12:10.900328 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133c57cb-1e04-4fea-9178-c30ee8065495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:12:10.974860 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-133c57cb-1e04-4fea-9178-c30ee8065495 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:12:10.976155 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-133c57cb-1e04-4fea-9178-c30ee8065495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:12:11.002858 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:12:11.072335 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 703/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 22 19:12:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:12:11.222948 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e300d23-4395-4002-bd45-06f5b8e560f7 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:12:11.224551 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-6e300d23-4395-4002-bd45-06f5b8e560f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:12:11.368968 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e300d23-4395-4002-bd45-06f5b8e560f7 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:12:11.409297 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:12:11.481852 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b060245e-e86d-49a8-af0f-0998f020e136 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f43e2756b746a6825ca1a06b7d7c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:12:11.482471 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b060245e-e86d-49a8-af0f-0998f020e136 service nova] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:12:11.482749 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b060245e-e86d-49a8-af0f-0998f020e136 service nova] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:12:11.483019 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b060245e-e86d-49a8-af0f-0998f020e136 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:12:11.485161 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-b060245e-e86d-49a8-af0f-0998f020e136 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:12:11.572987 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b060245e-e86d-49a8-af0f-0998f020e136 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:12:11.574581 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b060245e-e86d-49a8-af0f-0998f020e136 service nova] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:12:11.592452 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 716/1419] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:12:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:12:12.476859 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9877b0e6-4885-47a2-9075-75de63a4445d None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:12:12.477405 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9877b0e6-4885-47a2-9075-75de63a4445d None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:12:12.477405 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9877b0e6-4885-47a2-9075-75de63a4445d None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:12:12.479063 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 704/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 22 19:12:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:12:12.483925 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f709caad-c1cb-4016-86fa-6f5ba7b5ce89 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:12:12.484349 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f709caad-c1cb-4016-86fa-6f5ba7b5ce89 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:12:12.484453 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f709caad-c1cb-4016-86fa-6f5ba7b5ce89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:12:12.538119 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:12:12.559559 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-f709caad-c1cb-4016-86fa-6f5ba7b5ce89 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:12:12.560800 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-f709caad-c1cb-4016-86fa-6f5ba7b5ce89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:12:12.657004 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 717/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 22 19:12:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:12:12.807881 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9877b0e6-4885-47a2-9075-75de63a4445d None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:12:12.809349 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-9877b0e6-4885-47a2-9075-75de63a4445d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:12:12.955294 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9877b0e6-4885-47a2-9075-75de63a4445d None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:12:12.967367 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:12:13.068877 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48cc505-3092-4a35-b90b-987dacd0d785 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f43e2756b746a6825ca1a06b7d7c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:12:13.069395 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48cc505-3092-4a35-b90b-987dacd0d785 service nova] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:12:13.069673 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48cc505-3092-4a35-b90b-987dacd0d785 service nova] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:12:13.069945 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48cc505-3092-4a35-b90b-987dacd0d785 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:12:13.071586 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-c48cc505-3092-4a35-b90b-987dacd0d785 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:12:13.161319 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48cc505-3092-4a35-b90b-987dacd0d785 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:12:13.162898 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48cc505-3092-4a35-b90b-987dacd0d785 service nova] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:12:13.180492 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 705/1422] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:12:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:12:14.033137 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59761251-4739-460b-b6de-bb300c2a56a2 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:12:14.033724 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59761251-4739-460b-b6de-bb300c2a56a2 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:12:14.033724 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59761251-4739-460b-b6de-bb300c2a56a2 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:12:14.035054 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 718/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 22 19:12:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:12:14.040128 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86625728-ef3f-45b8-996f-61d5f0c4bb09 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:12:14.040542 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86625728-ef3f-45b8-996f-61d5f0c4bb09 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:12:14.040740 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86625728-ef3f-45b8-996f-61d5f0c4bb09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:12:14.060859 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:12:14.118021 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-86625728-ef3f-45b8-996f-61d5f0c4bb09 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:12:14.119230 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-86625728-ef3f-45b8-996f-61d5f0c4bb09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:12:14.213712 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 706/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 22 19:12:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:12:14.363734 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59761251-4739-460b-b6de-bb300c2a56a2 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:12:14.365207 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-59761251-4739-460b-b6de-bb300c2a56a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:12:14.475544 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:12:14.512190 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59761251-4739-460b-b6de-bb300c2a56a2 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:12:14.623590 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea394cbf-0069-4974-93a8-f30b8aad6b80 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f43e2756b746a6825ca1a06b7d7c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:12:14.624231 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea394cbf-0069-4974-93a8-f30b8aad6b80 service nova] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:12:14.624508 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea394cbf-0069-4974-93a8-f30b8aad6b80 service nova] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:12:14.624778 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea394cbf-0069-4974-93a8-f30b8aad6b80 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:12:14.626397 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-ea394cbf-0069-4974-93a8-f30b8aad6b80 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:12:14.700997 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:12:14.715399 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea394cbf-0069-4974-93a8-f30b8aad6b80 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:12:14.717038 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea394cbf-0069-4974-93a8-f30b8aad6b80 service nova] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:12:14.735738 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 719/1425] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:12:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:12:15.593852 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e98649-d574-4aff-9be2-2d899b76b4f2 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:12:15.594430 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e98649-d574-4aff-9be2-2d899b76b4f2 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:12:15.594430 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e98649-d574-4aff-9be2-2d899b76b4f2 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:12:15.595779 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 707/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 22 19:12:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:12:15.600799 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8734bec9-9399-4623-bee5-ea459d4d897f None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:12:15.601258 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8734bec9-9399-4623-bee5-ea459d4d897f None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:12:15.601358 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8734bec9-9399-4623-bee5-ea459d4d897f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:12:15.677200 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-8734bec9-9399-4623-bee5-ea459d4d897f None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:12:15.678419 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-8734bec9-9399-4623-bee5-ea459d4d897f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:12:15.774086 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 720/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 22 19:12:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:12:15.925663 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e98649-d574-4aff-9be2-2d899b76b4f2 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:12:15.927100 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-f5e98649-d574-4aff-9be2-2d899b76b4f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:12:16.008882 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:12:16.070092 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e98649-d574-4aff-9be2-2d899b76b4f2 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:12:16.183998 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f676773-5736-492e-ae35-583adfddb46c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f43e2756b746a6825ca1a06b7d7c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:12:16.184602 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f676773-5736-492e-ae35-583adfddb46c service nova] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:12:16.184877 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f676773-5736-492e-ae35-583adfddb46c service nova] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:12:16.185145 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f676773-5736-492e-ae35-583adfddb46c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:12:16.186770 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-8f676773-5736-492e-ae35-583adfddb46c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:12:16.234569 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:12:16.278620 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f676773-5736-492e-ae35-583adfddb46c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:12:16.280769 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f676773-5736-492e-ae35-583adfddb46c service nova] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:12:16.298824 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 708/1428] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:12:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:12:17.092214 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ba60c7-287b-41fa-be87-92f1f60ec433 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:12:17.092784 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ba60c7-287b-41fa-be87-92f1f60ec433 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:12:17.092784 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ba60c7-287b-41fa-be87-92f1f60ec433 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:12:17.094173 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 721/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 22 19:12:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:12:17.099161 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd61d04-884a-447e-b255-8142bd380a55 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:12:17.099600 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd61d04-884a-447e-b255-8142bd380a55 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:12:17.099745 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd61d04-884a-447e-b255-8142bd380a55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:12:17.174543 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-6bd61d04-884a-447e-b255-8142bd380a55 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:12:17.175786 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-6bd61d04-884a-447e-b255-8142bd380a55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:12:17.271917 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 709/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 22 19:12:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:12:17.422341 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ba60c7-287b-41fa-be87-92f1f60ec433 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:12:17.423966 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-59ba60c7-287b-41fa-be87-92f1f60ec433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:12:17.471701 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:12:17.567669 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ba60c7-287b-41fa-be87-92f1f60ec433 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:12:17.677287 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5bdb84-1ca5-4118-9d3a-286e37a98c1c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f43e2756b746a6825ca1a06b7d7c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:12:17.677898 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5bdb84-1ca5-4118-9d3a-286e37a98c1c service nova] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:12:17.678181 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5bdb84-1ca5-4118-9d3a-286e37a98c1c service nova] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:12:17.678455 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5bdb84-1ca5-4118-9d3a-286e37a98c1c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:12:17.680109 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-2d5bdb84-1ca5-4118-9d3a-286e37a98c1c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:12:17.687612 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:12:17.772109 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5bdb84-1ca5-4118-9d3a-286e37a98c1c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:12:17.773715 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5bdb84-1ca5-4118-9d3a-286e37a98c1c service nova] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:12:17.791809 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 722/1431] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:12:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:12:18.593938 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2e66a5-55de-45eb-a647-207dda1ac28f None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:12:18.594528 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2e66a5-55de-45eb-a647-207dda1ac28f None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:12:18.594528 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2e66a5-55de-45eb-a647-207dda1ac28f None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:12:18.595876 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 710/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 22 19:12:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:12:18.600951 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7312db-8bf1-4235-ada0-88fddae491f6 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:12:18.601402 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7312db-8bf1-4235-ada0-88fddae491f6 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:12:18.601507 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7312db-8bf1-4235-ada0-88fddae491f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:12:18.677681 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-2a7312db-8bf1-4235-ada0-88fddae491f6 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:12:18.678881 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-2a7312db-8bf1-4235-ada0-88fddae491f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:12:18.745541 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:12:18.774053 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 723/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 22 19:12:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:12:18.926676 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2e66a5-55de-45eb-a647-207dda1ac28f None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:12:18.928131 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-0c2e66a5-55de-45eb-a647-207dda1ac28f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:12:18.945406 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:12:19.074053 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2e66a5-55de-45eb-a647-207dda1ac28f None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:12:19.161574 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:12:19.187181 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee41c3d-7575-43dc-9ab5-168a2ac5a527 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f43e2756b746a6825ca1a06b7d7c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:12:19.187817 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee41c3d-7575-43dc-9ab5-168a2ac5a527 service nova] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:12:19.188117 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee41c3d-7575-43dc-9ab5-168a2ac5a527 service nova] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:12:19.188384 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee41c3d-7575-43dc-9ab5-168a2ac5a527 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:12:19.189980 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-6ee41c3d-7575-43dc-9ab5-168a2ac5a527 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:12:19.279266 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee41c3d-7575-43dc-9ab5-168a2ac5a527 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:12:19.281518 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee41c3d-7575-43dc-9ab5-168a2ac5a527 service nova] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:12:19.299112 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 711/1434] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:12:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:12:20.103909 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9195478b-c391-41a1-a1cd-8ddbbab7a381 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:12:20.104421 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9195478b-c391-41a1-a1cd-8ddbbab7a381 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:12:20.104536 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9195478b-c391-41a1-a1cd-8ddbbab7a381 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:12:20.105810 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 724/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 22 19:12:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:12:20.111339 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9308209-65b7-454f-bb67-3358f08c271c None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:12:20.111843 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9308209-65b7-454f-bb67-3358f08c271c None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:12:20.111943 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9308209-65b7-454f-bb67-3358f08c271c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:12:20.186007 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-a9308209-65b7-454f-bb67-3358f08c271c None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:12:20.187227 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-a9308209-65b7-454f-bb67-3358f08c271c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:12:20.228107 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:12:20.282616 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 712/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 22 19:12:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:12:20.431659 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9195478b-c391-41a1-a1cd-8ddbbab7a381 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:12:20.433191 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-9195478b-c391-41a1-a1cd-8ddbbab7a381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:12:20.579331 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9195478b-c391-41a1-a1cd-8ddbbab7a381 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:12:20.632151 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:12:20.692365 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bede17-548c-4709-9354-e33bbd567724 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f43e2756b746a6825ca1a06b7d7c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:12:20.692977 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bede17-548c-4709-9354-e33bbd567724 service nova] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:12:20.693271 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bede17-548c-4709-9354-e33bbd567724 service nova] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:12:20.693526 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bede17-548c-4709-9354-e33bbd567724 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:12:20.695134 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-34bede17-548c-4709-9354-e33bbd567724 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:12:20.785256 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bede17-548c-4709-9354-e33bbd567724 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:12:20.786857 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bede17-548c-4709-9354-e33bbd567724 service nova] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:12:20.804649 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 725/1437] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:12:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:12:21.600672 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70dc441-aa52-487e-9c2b-776300dc3e97 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:12:21.601275 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70dc441-aa52-487e-9c2b-776300dc3e97 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:12:21.601275 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70dc441-aa52-487e-9c2b-776300dc3e97 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:12:21.602588 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 713/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 22 19:12:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:12:21.607791 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07449a71-5f4b-4327-9467-e70ad362bee0 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:12:21.608794 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07449a71-5f4b-4327-9467-e70ad362bee0 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:12:21.608900 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07449a71-5f4b-4327-9467-e70ad362bee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:12:21.684853 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-07449a71-5f4b-4327-9467-e70ad362bee0 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:12:21.686069 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-07449a71-5f4b-4327-9467-e70ad362bee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:12:21.688198 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:12:21.781749 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 726/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 22 19:12:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:12:21.931518 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70dc441-aa52-487e-9c2b-776300dc3e97 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:12:21.933067 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-c70dc441-aa52-487e-9c2b-776300dc3e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:12:22.078617 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70dc441-aa52-487e-9c2b-776300dc3e97 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:12:22.104965 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:12:22.189758 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73e9aeb-df38-4bdf-b4be-756d381c87d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f43e2756b746a6825ca1a06b7d7c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:12:22.190359 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73e9aeb-df38-4bdf-b4be-756d381c87d2 service nova] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:12:22.190632 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73e9aeb-df38-4bdf-b4be-756d381c87d2 service nova] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:12:22.190897 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73e9aeb-df38-4bdf-b4be-756d381c87d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:12:22.192523 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-c73e9aeb-df38-4bdf-b4be-756d381c87d2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:12:22.284956 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73e9aeb-df38-4bdf-b4be-756d381c87d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:12:22.286522 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73e9aeb-df38-4bdf-b4be-756d381c87d2 service nova] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:12:22.304018 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 714/1440] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:12:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:12:24.118273 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b21bfd1-eec7-4913-b7a3-e2505d76c130 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:12:24.118841 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b21bfd1-eec7-4913-b7a3-e2505d76c130 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:12:24.118841 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b21bfd1-eec7-4913-b7a3-e2505d76c130 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:12:24.120205 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 727/1441] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 22 19:12:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:12:24.126125 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d985b28-c8c9-407d-aed4-96333a0d7d1d None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:12:24.126579 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d985b28-c8c9-407d-aed4-96333a0d7d1d None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:12:24.126698 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d985b28-c8c9-407d-aed4-96333a0d7d1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:12:24.167769 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:12:24.193535 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-4d985b28-c8c9-407d-aed4-96333a0d7d1d None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:12:24.194751 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-4d985b28-c8c9-407d-aed4-96333a0d7d1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:12:24.289084 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 715/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 22 19:12:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 19:12:24.571491 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b21bfd1-eec7-4913-b7a3-e2505d76c130 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:12:24.573063 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-3b21bfd1-eec7-4913-b7a3-e2505d76c130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:12:24.675021 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b21bfd1-eec7-4913-b7a3-e2505d76c130 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:12:24.707204 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:12:24.748882 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9738b52c-baca-474c-8ccd-082dc0fff68a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:12:24.749483 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9738b52c-baca-474c-8ccd-082dc0fff68a service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:12:24.749755 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9738b52c-baca-474c-8ccd-082dc0fff68a service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:12:24.750021 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9738b52c-baca-474c-8ccd-082dc0fff68a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:12:24.751599 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-9738b52c-baca-474c-8ccd-082dc0fff68a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:12:24.858350 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9738b52c-baca-474c-8ccd-082dc0fff68a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:12:24.859741 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9738b52c-baca-474c-8ccd-082dc0fff68a service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:12:24.877714 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 728/1443] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:12:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:12:26.923876 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e25bd12-26b0-4335-9bb6-ca6e60cdf8b0 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:12:26.924459 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e25bd12-26b0-4335-9bb6-ca6e60cdf8b0 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:12:26.924459 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e25bd12-26b0-4335-9bb6-ca6e60cdf8b0 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:12:26.925769 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 716/1444] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 22 19:12:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:12:26.930974 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f5b2fb-35a4-4058-9760-f1266de4724e None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:12:26.931456 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f5b2fb-35a4-4058-9760-f1266de4724e None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:12:26.931561 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f5b2fb-35a4-4058-9760-f1266de4724e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:12:26.997679 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-31f5b2fb-35a4-4058-9760-f1266de4724e None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:12:26.998925 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-31f5b2fb-35a4-4058-9760-f1266de4724e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:12:27.011426 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:12:27.098160 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 729/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 22 19:12:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:12:27.381079 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e25bd12-26b0-4335-9bb6-ca6e60cdf8b0 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:12:27.382642 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-5e25bd12-26b0-4335-9bb6-ca6e60cdf8b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:12:27.484615 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e25bd12-26b0-4335-9bb6-ca6e60cdf8b0 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:12:27.496233 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:12:27.558682 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4e2479-aae8-4eac-b902-555cefb2c0b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:12:27.559279 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4e2479-aae8-4eac-b902-555cefb2c0b5 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:12:27.559554 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4e2479-aae8-4eac-b902-555cefb2c0b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:12:27.559853 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4e2479-aae8-4eac-b902-555cefb2c0b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:12:27.561983 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-be4e2479-aae8-4eac-b902-555cefb2c0b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:12:27.667695 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4e2479-aae8-4eac-b902-555cefb2c0b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:12:27.669031 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4e2479-aae8-4eac-b902-555cefb2c0b5 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:12:27.687065 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 717/1446] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:12:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:12:29.981746 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f895e565-97ac-487a-b391-94ebd8dbfb8e None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:12:29.982303 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f895e565-97ac-487a-b391-94ebd8dbfb8e None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:12:29.982303 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f895e565-97ac-487a-b391-94ebd8dbfb8e None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:12:29.983686 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 730/1447] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 22 19:12:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:12:29.989021 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6c83d9-c86f-401c-93e3-b7efe7ae5d5d None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:12:29.989433 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6c83d9-c86f-401c-93e3-b7efe7ae5d5d None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:12:29.989565 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6c83d9-c86f-401c-93e3-b7efe7ae5d5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:12:30.038449 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:12:30.055761 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-5c6c83d9-c86f-401c-93e3-b7efe7ae5d5d None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:12:30.056956 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-5c6c83d9-c86f-401c-93e3-b7efe7ae5d5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:12:30.152807 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 718/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 22 19:12:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 19:12:30.432780 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f895e565-97ac-487a-b391-94ebd8dbfb8e None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:12:30.434323 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-f895e565-97ac-487a-b391-94ebd8dbfb8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:12:30.535344 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f895e565-97ac-487a-b391-94ebd8dbfb8e None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:12:30.575664 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:12:30.610069 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7edf93e-5931-47e6-9045-5c5ae5a1329c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:12:30.610673 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7edf93e-5931-47e6-9045-5c5ae5a1329c service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:12:30.610950 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7edf93e-5931-47e6-9045-5c5ae5a1329c service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:12:30.611220 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7edf93e-5931-47e6-9045-5c5ae5a1329c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:12:30.612854 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-c7edf93e-5931-47e6-9045-5c5ae5a1329c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:12:30.719344 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7edf93e-5931-47e6-9045-5c5ae5a1329c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:12:30.720820 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7edf93e-5931-47e6-9045-5c5ae5a1329c service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:12:30.739032 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 731/1449] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:12:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:12:32.756133 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754d1f83-3a34-4b4e-bc3e-79bae5f8cd7d None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:12:32.756708 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754d1f83-3a34-4b4e-bc3e-79bae5f8cd7d None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:12:32.756708 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754d1f83-3a34-4b4e-bc3e-79bae5f8cd7d None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:12:32.758011 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 719/1450] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 22 19:12:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:12:32.763352 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f7db8d-8cda-4343-b9b1-43dd13aa918b None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:12:32.764352 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f7db8d-8cda-4343-b9b1-43dd13aa918b None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:12:32.764489 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f7db8d-8cda-4343-b9b1-43dd13aa918b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:12:32.830523 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-73f7db8d-8cda-4343-b9b1-43dd13aa918b None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:12:32.831753 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-73f7db8d-8cda-4343-b9b1-43dd13aa918b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:12:32.851567 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:12:32.927653 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 732/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 22 19:12:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:12:33.218198 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754d1f83-3a34-4b4e-bc3e-79bae5f8cd7d None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:12:33.219803 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-754d1f83-3a34-4b4e-bc3e-79bae5f8cd7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:12:33.321809 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754d1f83-3a34-4b4e-bc3e-79bae5f8cd7d None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:12:33.342630 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:12:33.397228 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfdc21a-f990-4a4f-b128-7c9fc9ab35b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:12:33.397755 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfdc21a-f990-4a4f-b128-7c9fc9ab35b7 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:12:33.398028 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfdc21a-f990-4a4f-b128-7c9fc9ab35b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:12:33.398295 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfdc21a-f990-4a4f-b128-7c9fc9ab35b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:12:33.399920 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-0dfdc21a-f990-4a4f-b128-7c9fc9ab35b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:12:33.506213 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfdc21a-f990-4a4f-b128-7c9fc9ab35b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:12:33.507625 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfdc21a-f990-4a4f-b128-7c9fc9ab35b7 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:12:33.525668 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 720/1452] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:12:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:12:35.474514 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667e07ca-dab1-4f9f-bca6-bcc2a27c4521 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:12:35.475070 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667e07ca-dab1-4f9f-bca6-bcc2a27c4521 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:12:35.475070 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667e07ca-dab1-4f9f-bca6-bcc2a27c4521 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:12:35.476485 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 733/1453] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 22 19:12:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:12:35.481946 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f26929-05ba-426e-a783-fefcf0944411 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:12:35.482947 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f26929-05ba-426e-a783-fefcf0944411 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:12:35.483060 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f26929-05ba-426e-a783-fefcf0944411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:12:35.545898 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:12:35.548797 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-a3f26929-05ba-426e-a783-fefcf0944411 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:12:35.550629 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-a3f26929-05ba-426e-a783-fefcf0944411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:12:35.647139 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 721/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 22 19:12:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:12:35.932409 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667e07ca-dab1-4f9f-bca6-bcc2a27c4521 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:12:35.933952 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-667e07ca-dab1-4f9f-bca6-bcc2a27c4521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:12:36.037499 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667e07ca-dab1-4f9f-bca6-bcc2a27c4521 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:12:36.085910 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:12:36.112285 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9266b88-96a3-4927-8377-299677a7046d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:12:36.112896 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9266b88-96a3-4927-8377-299677a7046d service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:12:36.113176 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9266b88-96a3-4927-8377-299677a7046d service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:12:36.113445 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9266b88-96a3-4927-8377-299677a7046d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:12:36.115081 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-b9266b88-96a3-4927-8377-299677a7046d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:12:36.223553 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9266b88-96a3-4927-8377-299677a7046d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:12:36.225006 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9266b88-96a3-4927-8377-299677a7046d service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:12:36.242893 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 734/1455] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:12:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:12:38.202670 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1364675c-56e0-4429-b0fe-7266c426eb75 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:12:38.203243 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1364675c-56e0-4429-b0fe-7266c426eb75 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:12:38.203243 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1364675c-56e0-4429-b0fe-7266c426eb75 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:12:38.204592 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 722/1456] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 22 19:12:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:12:38.210126 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c98e07-7d83-48cb-a606-18bb3e1ea479 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:12:38.210539 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c98e07-7d83-48cb-a606-18bb3e1ea479 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:12:38.210667 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c98e07-7d83-48cb-a606-18bb3e1ea479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:12:38.281014 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-61c98e07-7d83-48cb-a606-18bb3e1ea479 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:12:38.282250 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-61c98e07-7d83-48cb-a606-18bb3e1ea479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:12:38.312844 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:12:38.381843 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 735/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 22 19:12:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:12:38.665730 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1364675c-56e0-4429-b0fe-7266c426eb75 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:12:38.667288 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-1364675c-56e0-4429-b0fe-7266c426eb75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:12:38.769384 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1364675c-56e0-4429-b0fe-7266c426eb75 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:12:38.797113 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:12:38.841925 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60039c4-b702-4448-9d30-b2a04c376f3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:12:38.842528 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60039c4-b702-4448-9d30-b2a04c376f3a service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:12:38.842804 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60039c4-b702-4448-9d30-b2a04c376f3a service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:12:38.843071 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60039c4-b702-4448-9d30-b2a04c376f3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:12:38.844703 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-b60039c4-b702-4448-9d30-b2a04c376f3a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:12:38.950737 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60039c4-b702-4448-9d30-b2a04c376f3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:12:38.952178 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60039c4-b702-4448-9d30-b2a04c376f3a service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:12:38.970076 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 723/1458] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:12:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:12:40.925911 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b5a739-ee6c-4174-9546-bc20ff3c0170 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:12:40.926500 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b5a739-ee6c-4174-9546-bc20ff3c0170 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:12:40.926500 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b5a739-ee6c-4174-9546-bc20ff3c0170 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:12:40.927820 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 736/1459] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 22 19:12:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:12:40.933166 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa61bad-839a-471c-b54a-330aee614bbf None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:12:40.933583 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa61bad-839a-471c-b54a-330aee614bbf None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:12:40.933738 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa61bad-839a-471c-b54a-330aee614bbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:12:40.999200 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-4aa61bad-839a-471c-b54a-330aee614bbf None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:12:41.000432 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-4aa61bad-839a-471c-b54a-330aee614bbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:12:41.008529 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:12:41.095967 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 724/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 22 19:12:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:12:41.376911 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b5a739-ee6c-4174-9546-bc20ff3c0170 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:12:41.378492 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-30b5a739-ee6c-4174-9546-bc20ff3c0170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:12:41.481316 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b5a739-ee6c-4174-9546-bc20ff3c0170 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:12:41.535660 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:12:41.554697 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d7443c-30d7-45b9-a7b1-6b7361dece33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:12:41.555301 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d7443c-30d7-45b9-a7b1-6b7361dece33 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:12:41.555580 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d7443c-30d7-45b9-a7b1-6b7361dece33 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:12:41.555891 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d7443c-30d7-45b9-a7b1-6b7361dece33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:12:41.557496 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-04d7443c-30d7-45b9-a7b1-6b7361dece33 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:12:41.665827 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d7443c-30d7-45b9-a7b1-6b7361dece33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:12:41.667183 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d7443c-30d7-45b9-a7b1-6b7361dece33 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:12:41.685037 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 737/1461] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:12:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:12:43.619811 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0083481f-3ef6-4a02-b98d-c9a1cdf09b95 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:12:43.620412 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0083481f-3ef6-4a02-b98d-c9a1cdf09b95 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:12:43.620412 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0083481f-3ef6-4a02-b98d-c9a1cdf09b95 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:12:43.621719 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 725/1462] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 22 19:12:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:12:43.626977 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3427b2ed-4154-4efc-9b11-6fc5788dc62b None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:12:43.627384 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3427b2ed-4154-4efc-9b11-6fc5788dc62b None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:12:43.627509 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3427b2ed-4154-4efc-9b11-6fc5788dc62b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:12:43.692209 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-3427b2ed-4154-4efc-9b11-6fc5788dc62b None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:12:43.693428 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-3427b2ed-4154-4efc-9b11-6fc5788dc62b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:12:43.731881 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:12:43.789757 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 738/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 22 19:12:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:12:44.070674 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0083481f-3ef6-4a02-b98d-c9a1cdf09b95 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:12:44.072271 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-0083481f-3ef6-4a02-b98d-c9a1cdf09b95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:12:44.175552 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0083481f-3ef6-4a02-b98d-c9a1cdf09b95 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:12:44.330525 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:12:44.368125 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e458229c-4615-437c-8c23-946761d18192 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:12:44.368729 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e458229c-4615-437c-8c23-946761d18192 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:12:44.369002 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e458229c-4615-437c-8c23-946761d18192 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:12:44.369266 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e458229c-4615-437c-8c23-946761d18192 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:12:44.370860 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-e458229c-4615-437c-8c23-946761d18192 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:12:44.477363 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e458229c-4615-437c-8c23-946761d18192 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:12:44.478778 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e458229c-4615-437c-8c23-946761d18192 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:12:44.496780 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 726/1464] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:12:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:12:46.434178 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb293952-6230-416b-afe9-cb4819e202b8 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:12:46.434761 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb293952-6230-416b-afe9-cb4819e202b8 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:12:46.434761 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb293952-6230-416b-afe9-cb4819e202b8 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:12:46.436139 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 739/1465] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 22 19:12:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:12:46.441357 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7348213d-b443-449b-89cc-3b61f4e75e04 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:12:46.441791 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7348213d-b443-449b-89cc-3b61f4e75e04 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:12:46.441896 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7348213d-b443-449b-89cc-3b61f4e75e04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:12:46.505873 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-7348213d-b443-449b-89cc-3b61f4e75e04 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:12:46.507066 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-7348213d-b443-449b-89cc-3b61f4e75e04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:12:46.524466 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:12:46.603065 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 727/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 22 19:12:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:12:46.882491 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb293952-6230-416b-afe9-cb4819e202b8 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:12:46.884050 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-cb293952-6230-416b-afe9-cb4819e202b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:12:46.990451 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb293952-6230-416b-afe9-cb4819e202b8 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:12:47.056247 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:12:47.063787 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62eb0b77-651d-45c7-b0d1-3ea25c9def86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:12:47.064474 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eb0b77-651d-45c7-b0d1-3ea25c9def86 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:12:47.065659 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eb0b77-651d-45c7-b0d1-3ea25c9def86 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:12:47.065941 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eb0b77-651d-45c7-b0d1-3ea25c9def86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:12:47.067624 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-62eb0b77-651d-45c7-b0d1-3ea25c9def86 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:12:47.176319 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62eb0b77-651d-45c7-b0d1-3ea25c9def86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:12:47.177746 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62eb0b77-651d-45c7-b0d1-3ea25c9def86 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:12:47.196065 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 740/1467] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:12:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:12:49.112442 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cd742e-9fe2-44bc-b324-e0ac14f33ef9 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:12:49.113027 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cd742e-9fe2-44bc-b324-e0ac14f33ef9 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:12:49.113027 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cd742e-9fe2-44bc-b324-e0ac14f33ef9 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:12:49.114345 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 728/1468] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 22 19:12:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:12:49.119918 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1551d414-dac7-44c5-99c1-9184886a4cc3 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:12:49.120368 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1551d414-dac7-44c5-99c1-9184886a4cc3 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:12:49.120495 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1551d414-dac7-44c5-99c1-9184886a4cc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:12:49.186412 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-1551d414-dac7-44c5-99c1-9184886a4cc3 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:12:49.187662 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-1551d414-dac7-44c5-99c1-9184886a4cc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:12:49.235342 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:12:49.284377 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 741/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 22 19:12:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:12:49.566119 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cd742e-9fe2-44bc-b324-e0ac14f33ef9 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:12:49.567697 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-65cd742e-9fe2-44bc-b324-e0ac14f33ef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:12:49.670230 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cd742e-9fe2-44bc-b324-e0ac14f33ef9 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:12:49.718292 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:12:49.744694 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c5caa6-3d73-4459-aae1-26df17b0650c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:12:49.745295 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c5caa6-3d73-4459-aae1-26df17b0650c service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:12:49.745571 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c5caa6-3d73-4459-aae1-26df17b0650c service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:12:49.745860 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c5caa6-3d73-4459-aae1-26df17b0650c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:12:49.748026 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-70c5caa6-3d73-4459-aae1-26df17b0650c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:12:49.853805 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c5caa6-3d73-4459-aae1-26df17b0650c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:12:49.855177 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c5caa6-3d73-4459-aae1-26df17b0650c service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:12:49.874635 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 729/1470] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:12:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:12:59.282625 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=22 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:12:59.871833 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=66 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:13:09.236803 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e128bf90-2f4f-4eec-978a-c5c064eff147 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:13:09.237337 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e128bf90-2f4f-4eec-978a-c5c064eff147 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:13:09.237337 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e128bf90-2f4f-4eec-978a-c5c064eff147 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:13:09.238671 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 742/1471] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:13:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:13:09.243868 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a39180-bf6b-4e2e-8a85-ea232386e117 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:13:09.244336 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a39180-bf6b-4e2e-8a85-ea232386e117 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:13:09.244450 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a39180-bf6b-4e2e-8a85-ea232386e117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:13:09.310381 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-37a39180-bf6b-4e2e-8a85-ea232386e117 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:13:09.311600 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-37a39180-bf6b-4e2e-8a85-ea232386e117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:13:09.405309 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 730/1472] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:13:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:13:09.701336 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be154879-fad6-4679-bc87-e7531fece498 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:13:09.701622 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be154879-fad6-4679-bc87-e7531fece498 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:13:09.701896 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be154879-fad6-4679-bc87-e7531fece498 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:13:09.703236 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 743/1473] 10.222.0.61 () {54 vars in 967 bytes} [Mon Aug 22 19:13:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:13:09.708313 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be348a10-d7e7-4d64-9be7-9cc3f8c9e01c None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:13:09.708597 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be348a10-d7e7-4d64-9be7-9cc3f8c9e01c None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:13:09.708984 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be348a10-d7e7-4d64-9be7-9cc3f8c9e01c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:13:09.784456 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-be348a10-d7e7-4d64-9be7-9cc3f8c9e01c None None] MFA Rules not processed for user `fe15d6a79ed44e5396d13cd29ba160ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:13:09.785659 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-be348a10-d7e7-4d64-9be7-9cc3f8c9e01c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:13:09.806062 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:13:09.860869 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 731/1474] 10.222.0.61 () {58 vars in 1064 bytes} [Mon Aug 22 19:13:09 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:13:09.868446 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be154879-fad6-4679-bc87-e7531fece498 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:13:09.869965 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-be154879-fad6-4679-bc87-e7531fece498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:13:09.972997 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be154879-fad6-4679-bc87-e7531fece498 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:13:10.045122 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5efcae-e9e1-4960-809b-027ba2aede96 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d6a79ed44e5396d13cd29ba160ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:13:10.045726 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5efcae-e9e1-4960-809b-027ba2aede96 service glance] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:13:10.046010 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5efcae-e9e1-4960-809b-027ba2aede96 service glance] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:13:10.046281 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5efcae-e9e1-4960-809b-027ba2aede96 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:13:10.047896 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-6b5efcae-e9e1-4960-809b-027ba2aede96 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:13:10.100657 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:13:10.156397 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5efcae-e9e1-4960-809b-027ba2aede96 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:13:10.157833 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5efcae-e9e1-4960-809b-027ba2aede96 service glance] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:13:10.176152 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 744/1475] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:13:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:13:12.153124 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37755fbe-e66b-4483-ad39-5a1a2f57778e None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:13:12.153559 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37755fbe-e66b-4483-ad39-5a1a2f57778e None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:13:12.153663 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37755fbe-e66b-4483-ad39-5a1a2f57778e None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:13:12.155046 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 732/1476] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:13:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:13:12.160225 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513c1d05-8096-43fa-be03-5c41907d68ba None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:13:12.160656 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513c1d05-8096-43fa-be03-5c41907d68ba None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:13:12.160760 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513c1d05-8096-43fa-be03-5c41907d68ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:13:12.227215 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-513c1d05-8096-43fa-be03-5c41907d68ba None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:13:12.228455 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-513c1d05-8096-43fa-be03-5c41907d68ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:13:12.322703 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 745/1477] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:13:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:13:12.593748 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37755fbe-e66b-4483-ad39-5a1a2f57778e None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:13:12.595284 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-37755fbe-e66b-4483-ad39-5a1a2f57778e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:13:12.716937 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37755fbe-e66b-4483-ad39-5a1a2f57778e None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:13:12.765033 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:13:12.810137 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e620b36-8d63-4440-9575-c7870d35914e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:13:12.810745 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e620b36-8d63-4440-9575-c7870d35914e admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:13:12.811019 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e620b36-8d63-4440-9575-c7870d35914e admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:13:12.811288 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e620b36-8d63-4440-9575-c7870d35914e admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:13:12.811954 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e620b36-8d63-4440-9575-c7870d35914e admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:13:12.813656 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: WARNING py.warnings [None req-8e620b36-8d63-4440-9575-c7870d35914e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 22 19:13:12.813656 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: warnings.warn(msg) Aug 22 19:13:12.813656 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: Aug 22 19:13:12.814785 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e620b36-8d63-4440-9575-c7870d35914e admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:13:12.832879 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 733/1478] 10.222.0.61 () {56 vars in 1108 bytes} [Mon Aug 22 19:13:12 2022] GET /identity/v3/projects => generated 2553 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:13:14.749274 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aab350-601b-4ade-be44-67a1fb0b1bb6 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:13:14.749828 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aab350-601b-4ade-be44-67a1fb0b1bb6 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:13:14.749828 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aab350-601b-4ade-be44-67a1fb0b1bb6 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:13:14.751166 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 746/1479] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:13:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:13:14.756376 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e8ac64-e466-41fb-94ee-349eced7b940 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:13:14.756810 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e8ac64-e466-41fb-94ee-349eced7b940 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:13:14.756926 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e8ac64-e466-41fb-94ee-349eced7b940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:13:14.822706 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-f8e8ac64-e466-41fb-94ee-349eced7b940 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:13:14.824540 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-f8e8ac64-e466-41fb-94ee-349eced7b940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:13:14.920481 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 734/1480] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:13:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:13:15.212726 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aab350-601b-4ade-be44-67a1fb0b1bb6 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:13:15.215096 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-34aab350-601b-4ade-be44-67a1fb0b1bb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:13:15.225885 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:13:15.363314 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aab350-601b-4ade-be44-67a1fb0b1bb6 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:13:15.441711 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:13:15.475013 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06be49a8-819f-4ead-8dc7-2bca789f93c5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f43e2756b746a6825ca1a06b7d7c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:13:15.475895 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06be49a8-819f-4ead-8dc7-2bca789f93c5 service nova] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:13:15.476172 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06be49a8-819f-4ead-8dc7-2bca789f93c5 service nova] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:13:15.476439 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06be49a8-819f-4ead-8dc7-2bca789f93c5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:13:15.478017 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-06be49a8-819f-4ead-8dc7-2bca789f93c5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:13:15.564862 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06be49a8-819f-4ead-8dc7-2bca789f93c5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:13:15.566449 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06be49a8-819f-4ead-8dc7-2bca789f93c5 service nova] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:13:15.584755 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 747/1481] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:13:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:13:17.671337 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2b7e62-867e-4dd7-b7ce-7f4848e910c7 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:13:17.671985 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2b7e62-867e-4dd7-b7ce-7f4848e910c7 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:13:17.671985 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2b7e62-867e-4dd7-b7ce-7f4848e910c7 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:13:17.673290 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 735/1482] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:13:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:13:17.678710 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73faa540-18b1-47e7-aa08-927a87b7e92a None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:13:17.679147 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73faa540-18b1-47e7-aa08-927a87b7e92a None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:13:17.679245 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73faa540-18b1-47e7-aa08-927a87b7e92a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:13:17.750646 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-73faa540-18b1-47e7-aa08-927a87b7e92a None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:13:17.751911 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-73faa540-18b1-47e7-aa08-927a87b7e92a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:13:17.918627 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:13:17.968942 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 748/1483] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:13:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:13:18.254847 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2b7e62-867e-4dd7-b7ce-7f4848e910c7 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:13:18.256468 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-3b2b7e62-867e-4dd7-b7ce-7f4848e910c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:13:18.279155 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:13:18.362113 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2b7e62-867e-4dd7-b7ce-7f4848e910c7 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:13:18.436621 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6730277-4848-43e9-a4e7-846ae1080b8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:13:18.437222 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6730277-4848-43e9-a4e7-846ae1080b8d service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:13:18.437496 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6730277-4848-43e9-a4e7-846ae1080b8d service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:13:18.437763 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6730277-4848-43e9-a4e7-846ae1080b8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:13:18.439359 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-f6730277-4848-43e9-a4e7-846ae1080b8d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:13:18.511750 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:13:18.546991 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6730277-4848-43e9-a4e7-846ae1080b8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:13:18.548440 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6730277-4848-43e9-a4e7-846ae1080b8d service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:13:18.566361 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 736/1484] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:13:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:13:20.527505 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbe9cb1-5eb6-4ba8-aad0-9e3fc1b66853 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:13:20.528078 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbe9cb1-5eb6-4ba8-aad0-9e3fc1b66853 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:13:20.528196 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbe9cb1-5eb6-4ba8-aad0-9e3fc1b66853 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:13:20.529449 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 749/1485] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:13:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:13:20.534842 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b24880-850f-4aa0-90c5-c927ed3a344b None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:13:20.535294 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b24880-850f-4aa0-90c5-c927ed3a344b None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:13:20.535415 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b24880-850f-4aa0-90c5-c927ed3a344b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:13:20.601296 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-a9b24880-850f-4aa0-90c5-c927ed3a344b None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:13:20.602428 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-a9b24880-850f-4aa0-90c5-c927ed3a344b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:13:20.699306 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 737/1486] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:13:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:13:20.986260 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbe9cb1-5eb6-4ba8-aad0-9e3fc1b66853 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:13:20.987945 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-0cbe9cb1-5eb6-4ba8-aad0-9e3fc1b66853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:13:21.090252 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbe9cb1-5eb6-4ba8-aad0-9e3fc1b66853 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:13:21.162832 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:13:21.164836 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483e366f-479b-44c2-ba03-76ec81624b8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:13:21.165769 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483e366f-479b-44c2-ba03-76ec81624b8d service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:13:21.166232 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483e366f-479b-44c2-ba03-76ec81624b8d service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:13:21.166561 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483e366f-479b-44c2-ba03-76ec81624b8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:13:21.168957 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-483e366f-479b-44c2-ba03-76ec81624b8d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:13:21.275592 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483e366f-479b-44c2-ba03-76ec81624b8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:13:21.277621 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483e366f-479b-44c2-ba03-76ec81624b8d service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:13:21.295109 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 750/1487] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:13:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:13:23.276340 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2a1a7f-2fc2-498b-bd65-0a8e121530e7 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:13:23.276871 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2a1a7f-2fc2-498b-bd65-0a8e121530e7 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:13:23.276871 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2a1a7f-2fc2-498b-bd65-0a8e121530e7 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:13:23.278204 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 738/1488] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:13:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:13:23.283614 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02de0a99-8fb0-4e03-9c8a-469d3555f787 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:13:23.284063 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02de0a99-8fb0-4e03-9c8a-469d3555f787 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:13:23.284202 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02de0a99-8fb0-4e03-9c8a-469d3555f787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:13:23.356395 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-02de0a99-8fb0-4e03-9c8a-469d3555f787 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:13:23.357635 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-02de0a99-8fb0-4e03-9c8a-469d3555f787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:13:23.411138 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:13:23.455693 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 751/1489] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:13:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:13:23.747242 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2a1a7f-2fc2-498b-bd65-0a8e121530e7 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:13:23.748816 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-2d2a1a7f-2fc2-498b-bd65-0a8e121530e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:13:23.783629 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:13:23.852184 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2a1a7f-2fc2-498b-bd65-0a8e121530e7 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:13:23.924343 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885c66d2-cd9f-4605-af48-6ac194a3185f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:13:23.924945 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885c66d2-cd9f-4605-af48-6ac194a3185f service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:13:23.925223 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885c66d2-cd9f-4605-af48-6ac194a3185f service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:13:23.925495 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885c66d2-cd9f-4605-af48-6ac194a3185f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:13:23.927095 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-885c66d2-cd9f-4605-af48-6ac194a3185f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:13:24.008736 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:13:24.037838 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885c66d2-cd9f-4605-af48-6ac194a3185f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:13:24.039266 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885c66d2-cd9f-4605-af48-6ac194a3185f service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:13:24.056836 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 739/1490] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:13:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:13:26.145982 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e534b20-9851-4d1e-86a8-641fa6cacc27 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:13:26.146419 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e534b20-9851-4d1e-86a8-641fa6cacc27 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:13:26.146528 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e534b20-9851-4d1e-86a8-641fa6cacc27 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:13:26.147907 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 752/1491] 10.222.0.61 () {54 vars in 927 bytes} [Mon Aug 22 19:13:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:13:26.153350 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15804d8e-c92f-4780-94e8-562a0b0dc51d None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:13:26.153833 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15804d8e-c92f-4780-94e8-562a0b0dc51d None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:13:26.153953 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15804d8e-c92f-4780-94e8-562a0b0dc51d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:13:26.220795 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-15804d8e-c92f-4780-94e8-562a0b0dc51d None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:13:26.222005 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-15804d8e-c92f-4780-94e8-562a0b0dc51d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:13:26.315376 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 740/1492] 10.222.0.61 () {58 vars in 1024 bytes} [Mon Aug 22 19:13:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 19:13:26.604700 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e534b20-9851-4d1e-86a8-641fa6cacc27 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:13:26.606292 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-3e534b20-9851-4d1e-86a8-641fa6cacc27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:13:26.707124 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e534b20-9851-4d1e-86a8-641fa6cacc27 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:13:26.780437 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cc1f7f-000c-4646-a0b7-3ecb48fad936 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:13:26.781046 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cc1f7f-000c-4646-a0b7-3ecb48fad936 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:13:26.781323 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cc1f7f-000c-4646-a0b7-3ecb48fad936 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:13:26.781592 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cc1f7f-000c-4646-a0b7-3ecb48fad936 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:13:26.783182 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-20cc1f7f-000c-4646-a0b7-3ecb48fad936 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:13:26.792178 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:13:26.892284 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cc1f7f-000c-4646-a0b7-3ecb48fad936 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:13:26.893634 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cc1f7f-000c-4646-a0b7-3ecb48fad936 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:13:26.911282 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 753/1493] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:13:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:13:36.313647 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=82 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:13:36.909064 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=50 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:14:44.642524 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be8763-74db-4458-9040-9ab8a0bbc170 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:14:44.643129 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be8763-74db-4458-9040-9ab8a0bbc170 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:14:44.643129 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be8763-74db-4458-9040-9ab8a0bbc170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:14:44.717537 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-b5be8763-74db-4458-9040-9ab8a0bbc170 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:14:44.718727 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-b5be8763-74db-4458-9040-9ab8a0bbc170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:14:44.812627 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 741/1494] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:14:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:14:44.820783 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88965493-c345-4d68-85ad-1f9469b761cc None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:14:44.821296 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88965493-c345-4d68-85ad-1f9469b761cc None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:14:44.821403 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88965493-c345-4d68-85ad-1f9469b761cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:14:44.896157 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-88965493-c345-4d68-85ad-1f9469b761cc None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:14:44.897800 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-88965493-c345-4d68-85ad-1f9469b761cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:14:44.992654 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 754/1495] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:14:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:14:44.997342 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5be8763-74db-4458-9040-9ab8a0bbc170 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:14:44.998567 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-b5be8763-74db-4458-9040-9ab8a0bbc170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:14:45.009856 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5be8763-74db-4458-9040-9ab8a0bbc170 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:14:45.071727 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:14:45.106962 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984be1c8-83b8-458a-92e0-4351b7be0ba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:14:45.107561 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984be1c8-83b8-458a-92e0-4351b7be0ba3 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:14:45.107866 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984be1c8-83b8-458a-92e0-4351b7be0ba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:14:45.108142 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984be1c8-83b8-458a-92e0-4351b7be0ba3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:14:45.108781 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984be1c8-83b8-458a-92e0-4351b7be0ba3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:14:45.109077 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984be1c8-83b8-458a-92e0-4351b7be0ba3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:14:45.110088 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: WARNING py.warnings [None req-984be1c8-83b8-458a-92e0-4351b7be0ba3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 22 19:14:45.110088 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: warnings.warn(msg) Aug 22 19:14:45.110088 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: Aug 22 19:14:45.110964 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984be1c8-83b8-458a-92e0-4351b7be0ba3 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:14:45.124009 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 742/1496] 10.222.0.61 () {58 vars in 1153 bytes} [Mon Aug 22 19:14:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:14:45.132355 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88965493-c345-4d68-85ad-1f9469b761cc None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:14:45.141570 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88965493-c345-4d68-85ad-1f9469b761cc None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:14:45.202347 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:14:45.236985 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ba91fd-1b5e-48fc-abf2-211e6acbdcf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:14:45.237597 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ba91fd-1b5e-48fc-abf2-211e6acbdcf5 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:14:45.237878 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ba91fd-1b5e-48fc-abf2-211e6acbdcf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:14:45.238152 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ba91fd-1b5e-48fc-abf2-211e6acbdcf5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:14:45.238785 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ba91fd-1b5e-48fc-abf2-211e6acbdcf5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:14:45.240543 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: WARNING py.warnings [None req-64ba91fd-1b5e-48fc-abf2-211e6acbdcf5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 22 19:14:45.240543 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: warnings.warn(msg) Aug 22 19:14:45.240543 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: Aug 22 19:14:45.241732 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ba91fd-1b5e-48fc-abf2-211e6acbdcf5 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:14:45.295633 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 755/1497] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 19:14:45 2022] POST /identity/v3/projects => generated 377 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:14:45.299821 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984be1c8-83b8-458a-92e0-4351b7be0ba3 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:14:45.301341 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-984be1c8-83b8-458a-92e0-4351b7be0ba3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:14:45.333914 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984be1c8-83b8-458a-92e0-4351b7be0ba3 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:14:45.425142 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079bd55c-446a-44ca-ac3d-e694739db5aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:14:45.425742 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079bd55c-446a-44ca-ac3d-e694739db5aa admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:14:45.426016 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079bd55c-446a-44ca-ac3d-e694739db5aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:14:45.426283 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079bd55c-446a-44ca-ac3d-e694739db5aa admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:14:45.426912 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079bd55c-446a-44ca-ac3d-e694739db5aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:14:45.427965 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: WARNING py.warnings [None req-079bd55c-446a-44ca-ac3d-e694739db5aa admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 22 19:14:45.427965 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: warnings.warn(msg) Aug 22 19:14:45.427965 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: Aug 22 19:14:45.429164 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079bd55c-446a-44ca-ac3d-e694739db5aa admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:14:45.470818 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 743/1498] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 19:14:45 2022] POST /identity/v3/users => generated 289 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:14:45.475119 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ba91fd-1b5e-48fc-abf2-211e6acbdcf5 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:14:45.476685 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-64ba91fd-1b5e-48fc-abf2-211e6acbdcf5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:14:45.536983 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ba91fd-1b5e-48fc-abf2-211e6acbdcf5 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:14:45.621186 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:14:45.632460 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c47903-5ab8-4890-8a2d-3ca5033690df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:14:45.633056 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c47903-5ab8-4890-8a2d-3ca5033690df admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:14:45.633327 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c47903-5ab8-4890-8a2d-3ca5033690df admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:14:45.633590 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c47903-5ab8-4890-8a2d-3ca5033690df admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:14:45.634239 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c47903-5ab8-4890-8a2d-3ca5033690df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:14:45.635221 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: WARNING py.warnings [None req-39c47903-5ab8-4890-8a2d-3ca5033690df admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 22 19:14:45.635221 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: warnings.warn(msg) Aug 22 19:14:45.635221 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: Aug 22 19:14:45.636156 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c47903-5ab8-4890-8a2d-3ca5033690df admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:14:45.648708 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 756/1499] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:14:45 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:14:45.652682 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079bd55c-446a-44ca-ac3d-e694739db5aa admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:14:45.654227 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-079bd55c-446a-44ca-ac3d-e694739db5aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:14:45.666538 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079bd55c-446a-44ca-ac3d-e694739db5aa admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:14:45.691344 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:14:45.761858 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd1a146-a399-454b-9135-16fc77708719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:14:45.762390 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd1a146-a399-454b-9135-16fc77708719 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:14:45.762664 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd1a146-a399-454b-9135-16fc77708719 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:14:45.762929 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd1a146-a399-454b-9135-16fc77708719 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:14:45.763565 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd1a146-a399-454b-9135-16fc77708719 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:14:45.764599 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: WARNING py.warnings [None req-8bd1a146-a399-454b-9135-16fc77708719 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 22 19:14:45.764599 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: warnings.warn(msg) Aug 22 19:14:45.764599 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: Aug 22 19:14:45.765476 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd1a146-a399-454b-9135-16fc77708719 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:14:45.778126 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 744/1500] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:14:45 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:14:45.782298 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c47903-5ab8-4890-8a2d-3ca5033690df admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:14:45.783862 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-39c47903-5ab8-4890-8a2d-3ca5033690df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:14:45.817093 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c47903-5ab8-4890-8a2d-3ca5033690df admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:14:45.909676 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fddc58c-81a4-4930-b896-8031cd898e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:14:45.910330 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fddc58c-81a4-4930-b896-8031cd898e8c admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:14:45.910590 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fddc58c-81a4-4930-b896-8031cd898e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:14:45.910863 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fddc58c-81a4-4930-b896-8031cd898e8c admin admin] PATH_INFO: `/v3/projects/8f10d73282fd4e1282d50ab2393d3e8d/users/8a9111998a264494b3b8cb719428d9ec/roles/74a40bc819cb42eea7e966a4b2a016f1` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:14:45.932663 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fddc58c-81a4-4930-b896-8031cd898e8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f10d73282fd4e1282d50ab2393d3e8d, user_id=8a9111998a264494b3b8cb719428d9ec, role_id=74a40bc819cb42eea7e966a4b2a016f1)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:14:45.933678 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: WARNING py.warnings [None req-9fddc58c-81a4-4930-b896-8031cd898e8c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 22 19:14:45.933678 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: warnings.warn(msg) Aug 22 19:14:45.933678 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: Aug 22 19:14:45.935601 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fddc58c-81a4-4930-b896-8031cd898e8c admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:14:45.960422 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 757/1501] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:14:45 2022] PUT /identity/v3/projects/8f10d73282fd4e1282d50ab2393d3e8d/users/8a9111998a264494b3b8cb719428d9ec/roles/74a40bc819cb42eea7e966a4b2a016f1 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:14:45.972732 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddab365-f5f9-49b2-b61e-53dd917e30d3 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:14:45.973019 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddab365-f5f9-49b2-b61e-53dd917e30d3 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:14:45.973294 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddab365-f5f9-49b2-b61e-53dd917e30d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:14:46.030551 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-7ddab365-f5f9-49b2-b61e-53dd917e30d3 None None] MFA Rules not processed for user `8a9111998a264494b3b8cb719428d9ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:14:46.031723 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-7ddab365-f5f9-49b2-b61e-53dd917e30d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:14:46.107447 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 745/1502] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:14:45 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:14:46.120647 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e4188b-d288-4125-afe9-7dc9c25ec1f1 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:14:46.120934 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e4188b-d288-4125-afe9-7dc9c25ec1f1 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:14:46.121210 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e4188b-d288-4125-afe9-7dc9c25ec1f1 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:14:46.122562 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 758/1503] 10.222.0.61 () {54 vars in 964 bytes} [Mon Aug 22 19:14:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:14:46.127261 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1138bdf-1f6c-47cd-ad55-814816b3acb0 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:14:46.127543 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1138bdf-1f6c-47cd-ad55-814816b3acb0 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:14:46.127848 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1138bdf-1f6c-47cd-ad55-814816b3acb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:14:46.159409 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:14:46.203717 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-e1138bdf-1f6c-47cd-ad55-814816b3acb0 None None] MFA Rules not processed for user `56e79498873d4ed29a0e19bbea9b9214`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:14:46.204931 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-e1138bdf-1f6c-47cd-ad55-814816b3acb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:14:46.279587 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 746/1504] 10.222.0.61 () {58 vars in 1061 bytes} [Mon Aug 22 19:14:46 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:14:46.287617 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e4188b-d288-4125-afe9-7dc9c25ec1f1 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:14:46.289138 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-a7e4188b-d288-4125-afe9-7dc9c25ec1f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:14:46.339999 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:14:46.395274 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e4188b-d288-4125-afe9-7dc9c25ec1f1 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:14:46.467757 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eddffc-9a79-42d9-8f4b-b352f930befb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e79498873d4ed29a0e19bbea9b9214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:14:46.468363 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eddffc-9a79-42d9-8f4b-b352f930befb service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:14:46.468645 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eddffc-9a79-42d9-8f4b-b352f930befb service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:14:46.468916 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eddffc-9a79-42d9-8f4b-b352f930befb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:14:46.470520 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-f1eddffc-9a79-42d9-8f4b-b352f930befb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:14:46.567787 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eddffc-9a79-42d9-8f4b-b352f930befb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:14:46.569734 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eddffc-9a79-42d9-8f4b-b352f930befb service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:14:46.573209 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:14:46.588120 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 759/1505] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:14:46 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:14:46.632859 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d121c21-c2e2-43b4-841c-2f95664be24c None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:14:46.633150 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d121c21-c2e2-43b4-841c-2f95664be24c None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:14:46.633429 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d121c21-c2e2-43b4-841c-2f95664be24c None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:14:46.634687 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 747/1506] 10.222.0.61 () {52 vars in 830 bytes} [Mon Aug 22 19:14:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:14:46.638310 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eddffc-9a79-42d9-8f4b-b352f930befb service cinder] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:14:46.639521 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-f1eddffc-9a79-42d9-8f4b-b352f930befb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:14:46.738535 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eddffc-9a79-42d9-8f4b-b352f930befb service cinder] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:14:46.832091 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bccdd2-8ced-4137-b9d8-f7465feca38e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:14:46.832645 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bccdd2-8ced-4137-b9d8-f7465feca38e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:14:46.832926 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bccdd2-8ced-4137-b9d8-f7465feca38e admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:14:46.833201 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bccdd2-8ced-4137-b9d8-f7465feca38e admin admin] PATH_INFO: `/v3/users/8a9111998a264494b3b8cb719428d9ec` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:14:46.845689 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bccdd2-8ced-4137-b9d8-f7465feca38e admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a9111998a264494b3b8cb719428d9ec)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:14:46.846969 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: WARNING py.warnings [None req-99bccdd2-8ced-4137-b9d8-f7465feca38e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 22 19:14:46.846969 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: warnings.warn(msg) Aug 22 19:14:46.846969 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: Aug 22 19:14:46.848213 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bccdd2-8ced-4137-b9d8-f7465feca38e admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:14:46.857705 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:14:46.915268 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-99bccdd2-8ced-4137-b9d8-f7465feca38e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a9111998a264494b3b8cb719428d9ec'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:14:46.915962 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-99bccdd2-8ced-4137-b9d8-f7465feca38e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a9111998a264494b3b8cb719428d9ec'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:14:46.928899 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-99bccdd2-8ced-4137-b9d8-f7465feca38e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a9111998a264494b3b8cb719428d9ec'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:14:46.938084 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-99bccdd2-8ced-4137-b9d8-f7465feca38e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a9111998a264494b3b8cb719428d9ec'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:14:46.961320 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-99bccdd2-8ced-4137-b9d8-f7465feca38e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a9111998a264494b3b8cb719428d9ec'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:14:46.967075 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 760/1507] 10.222.0.61 () {58 vars in 1224 bytes} [Mon Aug 22 19:14:46 2022] DELETE /identity/v3/users/8a9111998a264494b3b8cb719428d9ec => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 19:14:46.971412 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a5e480-d300-42a2-a29b-97650eefab94 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:14:46.972159 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a5e480-d300-42a2-a29b-97650eefab94 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:14:46.972253 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a5e480-d300-42a2-a29b-97650eefab94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:14:47.028134 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-23a5e480-d300-42a2-a29b-97650eefab94 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:14:47.029348 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-23a5e480-d300-42a2-a29b-97650eefab94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:14:47.105768 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:14:47.118994 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 748/1508] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:14:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:14:47.128626 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bccdd2-8ced-4137-b9d8-f7465feca38e admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:14:47.129886 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-99bccdd2-8ced-4137-b9d8-f7465feca38e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:14:47.229410 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bccdd2-8ced-4137-b9d8-f7465feca38e admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:14:47.285664 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:14:47.307415 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97672faf-47df-483b-a051-7b70a992dc1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:14:47.308060 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97672faf-47df-483b-a051-7b70a992dc1f service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:14:47.308340 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97672faf-47df-483b-a051-7b70a992dc1f service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:14:47.308611 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97672faf-47df-483b-a051-7b70a992dc1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:14:47.310255 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-97672faf-47df-483b-a051-7b70a992dc1f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:14:47.417116 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97672faf-47df-483b-a051-7b70a992dc1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:14:47.418513 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97672faf-47df-483b-a051-7b70a992dc1f service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:14:47.436730 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 761/1509] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:14:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:14:47.784565 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a5e480-d300-42a2-a29b-97650eefab94 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:14:47.785833 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-23a5e480-d300-42a2-a29b-97650eefab94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:14:47.797239 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a5e480-d300-42a2-a29b-97650eefab94 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:14:47.893853 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e622951f-e626-407c-bca4-497f695055fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:14:47.894478 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e622951f-e626-407c-bca4-497f695055fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:14:47.894755 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e622951f-e626-407c-bca4-497f695055fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:14:47.895025 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e622951f-e626-407c-bca4-497f695055fd admin admin] PATH_INFO: `/v3/projects/8f10d73282fd4e1282d50ab2393d3e8d` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:14:47.904697 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e622951f-e626-407c-bca4-497f695055fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f10d73282fd4e1282d50ab2393d3e8d)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:14:47.905706 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: WARNING py.warnings [None req-e622951f-e626-407c-bca4-497f695055fd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 22 19:14:47.905706 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: warnings.warn(msg) Aug 22 19:14:47.905706 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: Aug 22 19:14:47.906893 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e622951f-e626-407c-bca4-497f695055fd admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:14:47.963164 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-e622951f-e626-407c-bca4-497f695055fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f10d73282fd4e1282d50ab2393d3e8d', 'request_id': 'req-e622951f-e626-407c-bca4-497f695055fd'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:14:47.972301 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-e622951f-e626-407c-bca4-497f695055fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f10d73282fd4e1282d50ab2393d3e8d', 'request_id': 'req-e622951f-e626-407c-bca4-497f695055fd'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:14:47.986969 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-e622951f-e626-407c-bca4-497f695055fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f10d73282fd4e1282d50ab2393d3e8d', 'request_id': 'req-e622951f-e626-407c-bca4-497f695055fd'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:14:48.010050 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-e622951f-e626-407c-bca4-497f695055fd admin admin] The token cache is being invalidate because project 8f10d73282fd4e1282d50ab2393d3e8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 19:14:48.010331 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-e622951f-e626-407c-bca4-497f695055fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:14:48.010746 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-e622951f-e626-407c-bca4-497f695055fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:14:48.012141 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 749/1510] 10.222.0.61 () {58 vars in 1233 bytes} [Mon Aug 22 19:14:47 2022] DELETE /identity/v3/projects/8f10d73282fd4e1282d50ab2393d3e8d => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:14:57.434460 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=62 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:14:58.007853 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:16:17.069687 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab7c54b-2e81-470a-a313-bab06aac7665 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:16:17.070259 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab7c54b-2e81-470a-a313-bab06aac7665 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:16:17.070259 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab7c54b-2e81-470a-a313-bab06aac7665 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:16:17.071576 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 762/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 22 19:16:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:16:17.076457 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcb0e98-caf5-4322-98c5-113a14837c9e None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:16:17.077012 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcb0e98-caf5-4322-98c5-113a14837c9e None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:16:17.077012 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcb0e98-caf5-4322-98c5-113a14837c9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:16:17.157908 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-dfcb0e98-caf5-4322-98c5-113a14837c9e None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:16:17.159129 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-dfcb0e98-caf5-4322-98c5-113a14837c9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:16:17.261594 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 750/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 22 19:16:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:16:17.415337 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab7c54b-2e81-470a-a313-bab06aac7665 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:16:17.417440 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-7ab7c54b-2e81-470a-a313-bab06aac7665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:16:17.536949 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab7c54b-2e81-470a-a313-bab06aac7665 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:16:17.543119 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf78585-7852-46d3-bb20-5325b906d910 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f43e2756b746a6825ca1a06b7d7c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:16:17.543800 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf78585-7852-46d3-bb20-5325b906d910 service nova] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:16:17.544086 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf78585-7852-46d3-bb20-5325b906d910 service nova] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:16:17.544364 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf78585-7852-46d3-bb20-5325b906d910 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:16:17.553041 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf78585-7852-46d3-bb20-5325b906d910 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:16:17.555449 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf78585-7852-46d3-bb20-5325b906d910 service nova] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:16:17.566234 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 763/1513] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:16:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:16:18.386135 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdf6b5f-f5e2-40d0-8f67-1f621f418f76 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:16:18.386630 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdf6b5f-f5e2-40d0-8f67-1f621f418f76 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:16:18.386743 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdf6b5f-f5e2-40d0-8f67-1f621f418f76 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:16:18.388140 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 751/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 22 19:16:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:16:18.392815 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d044a479-d7a3-4b6d-98e2-9511ab82424e None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:16:18.393246 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d044a479-d7a3-4b6d-98e2-9511ab82424e None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:16:18.393353 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d044a479-d7a3-4b6d-98e2-9511ab82424e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:16:18.413159 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-d044a479-d7a3-4b6d-98e2-9511ab82424e None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:16:18.414291 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-d044a479-d7a3-4b6d-98e2-9511ab82424e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:16:18.422054 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 764/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 22 19:16:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:16:18.572582 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdf6b5f-f5e2-40d0-8f67-1f621f418f76 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:16:18.586766 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdf6b5f-f5e2-40d0-8f67-1f621f418f76 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:16:18.592123 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c64328-d29f-4aed-b489-f888731a4981 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f43e2756b746a6825ca1a06b7d7c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:16:18.592713 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c64328-d29f-4aed-b489-f888731a4981 service nova] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:16:18.592986 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c64328-d29f-4aed-b489-f888731a4981 service nova] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:16:18.593254 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c64328-d29f-4aed-b489-f888731a4981 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:16:18.601414 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c64328-d29f-4aed-b489-f888731a4981 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:16:18.602974 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c64328-d29f-4aed-b489-f888731a4981 service nova] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:16:18.612590 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 752/1516] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:16:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:16:19.242787 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de51e096-7ada-42e6-95cc-640d16f59f37 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:16:19.243065 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de51e096-7ada-42e6-95cc-640d16f59f37 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:16:19.243335 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de51e096-7ada-42e6-95cc-640d16f59f37 None None] PATH_INFO: `/v3` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:16:19.244656 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 765/1517] 10.222.0.16 () {54 vars in 925 bytes} [Mon Aug 22 19:16:19 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:16:19.251349 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19157bc3-9535-4336-afc0-008e1bac3344 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:16:19.251634 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19157bc3-9535-4336-afc0-008e1bac3344 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:16:19.251955 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19157bc3-9535-4336-afc0-008e1bac3344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:16:19.310069 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-19157bc3-9535-4336-afc0-008e1bac3344 None None] MFA Rules not processed for user `eef0f73297e04d62bc22acb790fad6ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:16:19.311266 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-19157bc3-9535-4336-afc0-008e1bac3344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:16:19.327752 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:16:19.381828 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 753/1518] 10.222.0.16 () {58 vars in 1015 bytes} [Mon Aug 22 19:16:19 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:16:28.307398 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9053b9a2-aacb-4d72-8b1d-1ce86bd07a91 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:16:28.307986 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9053b9a2-aacb-4d72-8b1d-1ce86bd07a91 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:16:28.307986 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9053b9a2-aacb-4d72-8b1d-1ce86bd07a91 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:16:28.309223 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 766/1519] 10.222.0.61 () {54 vars in 961 bytes} [Mon Aug 22 19:16:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:16:28.314005 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61716202-4b1c-4b82-9fc0-1efb85ca9627 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:16:28.314449 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61716202-4b1c-4b82-9fc0-1efb85ca9627 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:16:28.314556 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61716202-4b1c-4b82-9fc0-1efb85ca9627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:16:28.333435 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-61716202-4b1c-4b82-9fc0-1efb85ca9627 None None] MFA Rules not processed for user `eef0f73297e04d62bc22acb790fad6ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:16:28.334641 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-61716202-4b1c-4b82-9fc0-1efb85ca9627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:16:28.343583 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 754/1520] 10.222.0.61 () {58 vars in 1058 bytes} [Mon Aug 22 19:16:28 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:16:28.351627 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9053b9a2-aacb-4d72-8b1d-1ce86bd07a91 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:16:28.365061 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9053b9a2-aacb-4d72-8b1d-1ce86bd07a91 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:16:28.370011 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bd0beb-b733-4ab8-baeb-8b6e9bb95409 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eef0f73297e04d62bc22acb790fad6ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:16:28.370597 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bd0beb-b733-4ab8-baeb-8b6e9bb95409 service placement] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:16:28.370876 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bd0beb-b733-4ab8-baeb-8b6e9bb95409 service placement] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:16:28.371154 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bd0beb-b733-4ab8-baeb-8b6e9bb95409 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:16:28.379246 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bd0beb-b733-4ab8-baeb-8b6e9bb95409 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:16:28.380864 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bd0beb-b733-4ab8-baeb-8b6e9bb95409 service placement] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:16:28.390389 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 767/1521] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:16:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:16:29.852940 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a8fd4b-e098-4a1b-97fb-8babceace4be None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:16:29.853533 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a8fd4b-e098-4a1b-97fb-8babceace4be None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:16:29.853533 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a8fd4b-e098-4a1b-97fb-8babceace4be None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:16:29.854827 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 755/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 22 19:16:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:16:29.859660 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45967a0e-02ff-45ac-bb96-4cb9309017fc None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:16:29.860093 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45967a0e-02ff-45ac-bb96-4cb9309017fc None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:16:29.860201 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45967a0e-02ff-45ac-bb96-4cb9309017fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:16:29.879022 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-45967a0e-02ff-45ac-bb96-4cb9309017fc None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:16:29.880318 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-45967a0e-02ff-45ac-bb96-4cb9309017fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:16:29.888213 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 768/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 22 19:16:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:16:30.041308 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a8fd4b-e098-4a1b-97fb-8babceace4be None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:16:30.055704 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a8fd4b-e098-4a1b-97fb-8babceace4be None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:16:30.061163 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4744ed06-7c67-4f6d-8fcb-84557ee6e541 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f43e2756b746a6825ca1a06b7d7c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:16:30.061793 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4744ed06-7c67-4f6d-8fcb-84557ee6e541 service nova] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:16:30.062071 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4744ed06-7c67-4f6d-8fcb-84557ee6e541 service nova] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:16:30.062341 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4744ed06-7c67-4f6d-8fcb-84557ee6e541 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:16:30.070360 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4744ed06-7c67-4f6d-8fcb-84557ee6e541 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:16:30.071953 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4744ed06-7c67-4f6d-8fcb-84557ee6e541 service nova] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:16:30.081161 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 756/1524] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:16:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:16:30.895902 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0868db51-d307-43c9-af55-dea83d45522c None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:16:30.896509 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0868db51-d307-43c9-af55-dea83d45522c None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:16:30.896509 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0868db51-d307-43c9-af55-dea83d45522c None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:16:30.897858 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 769/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 22 19:16:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:16:30.903039 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a1a3bf-3bf9-40b6-9791-efa49fd1080a None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:16:30.903483 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a1a3bf-3bf9-40b6-9791-efa49fd1080a None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:16:30.903728 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a1a3bf-3bf9-40b6-9791-efa49fd1080a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:16:30.922698 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-33a1a3bf-3bf9-40b6-9791-efa49fd1080a None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:16:30.923943 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-33a1a3bf-3bf9-40b6-9791-efa49fd1080a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:16:30.931894 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 757/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 22 19:16:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:16:31.086093 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0868db51-d307-43c9-af55-dea83d45522c None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:16:31.101742 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0868db51-d307-43c9-af55-dea83d45522c None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:16:31.107118 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8d23b0-10bc-423d-9165-25df85386381 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f43e2756b746a6825ca1a06b7d7c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:16:31.107753 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8d23b0-10bc-423d-9165-25df85386381 service nova] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:16:31.108036 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8d23b0-10bc-423d-9165-25df85386381 service nova] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:16:31.108303 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8d23b0-10bc-423d-9165-25df85386381 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:16:31.116937 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8d23b0-10bc-423d-9165-25df85386381 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:16:31.118522 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8d23b0-10bc-423d-9165-25df85386381 service nova] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:16:31.127902 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 770/1527] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:16:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:16:34.805162 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086fc000-3d4e-4b76-9ff9-6d6995f74e99 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:16:34.805745 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086fc000-3d4e-4b76-9ff9-6d6995f74e99 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:16:34.805745 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086fc000-3d4e-4b76-9ff9-6d6995f74e99 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:16:34.807055 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 758/1528] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 22 19:16:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:16:34.812224 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df079e9-2661-40ef-927b-387cb3595aa4 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:16:34.812693 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df079e9-2661-40ef-927b-387cb3595aa4 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:16:34.812803 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df079e9-2661-40ef-927b-387cb3595aa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:16:34.832428 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-6df079e9-2661-40ef-927b-387cb3595aa4 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:16:34.833626 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-6df079e9-2661-40ef-927b-387cb3595aa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:16:34.841983 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 771/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 22 19:16:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:16:35.128694 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086fc000-3d4e-4b76-9ff9-6d6995f74e99 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:16:35.130423 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-086fc000-3d4e-4b76-9ff9-6d6995f74e99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:16:35.210508 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086fc000-3d4e-4b76-9ff9-6d6995f74e99 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:16:35.214101 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bc2065-a8b2-42cf-88df-f3f23831bb2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:16:35.214718 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bc2065-a8b2-42cf-88df-f3f23831bb2e service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:16:35.214991 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bc2065-a8b2-42cf-88df-f3f23831bb2e service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:16:35.215268 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bc2065-a8b2-42cf-88df-f3f23831bb2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:16:35.224631 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bc2065-a8b2-42cf-88df-f3f23831bb2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:16:35.226082 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bc2065-a8b2-42cf-88df-f3f23831bb2e service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:16:35.235603 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 759/1530] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:16:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:16:37.154854 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf32af0-096b-449c-b737-b243fe4788df None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:16:37.155287 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf32af0-096b-449c-b737-b243fe4788df None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:16:37.155404 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf32af0-096b-449c-b737-b243fe4788df None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:16:37.156810 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 772/1531] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 22 19:16:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:16:37.162111 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7389fa-fffa-43e4-bcab-c536ca1130a2 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:16:37.162539 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7389fa-fffa-43e4-bcab-c536ca1130a2 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:16:37.162640 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7389fa-fffa-43e4-bcab-c536ca1130a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:16:37.181777 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-cb7389fa-fffa-43e4-bcab-c536ca1130a2 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:16:37.182978 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-cb7389fa-fffa-43e4-bcab-c536ca1130a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:16:37.190837 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 760/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 22 19:16:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:16:37.476541 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf32af0-096b-449c-b737-b243fe4788df None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:16:37.478254 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-baf32af0-096b-449c-b737-b243fe4788df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:16:37.485872 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=75 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:16:37.493038 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf32af0-096b-449c-b737-b243fe4788df None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:16:37.496577 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c868c0bb-8e59-4949-aa9c-dd9d386a02fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:16:37.497178 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c868c0bb-8e59-4949-aa9c-dd9d386a02fe service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:16:37.497459 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c868c0bb-8e59-4949-aa9c-dd9d386a02fe service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:16:37.497759 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c868c0bb-8e59-4949-aa9c-dd9d386a02fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:16:37.506631 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c868c0bb-8e59-4949-aa9c-dd9d386a02fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:16:37.508664 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c868c0bb-8e59-4949-aa9c-dd9d386a02fe service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:16:37.518014 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 773/1533] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:16:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:16:40.987449 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aff59d-6f97-4ea9-a373-bec8c9f29448 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:16:40.988091 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aff59d-6f97-4ea9-a373-bec8c9f29448 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:16:40.988091 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aff59d-6f97-4ea9-a373-bec8c9f29448 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:16:40.989403 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 761/1534] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 22 19:16:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:16:40.995043 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afff3a8a-bfa6-4b37-b042-653eab2c24c3 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:16:40.995511 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afff3a8a-bfa6-4b37-b042-653eab2c24c3 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:16:40.995610 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afff3a8a-bfa6-4b37-b042-653eab2c24c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:16:41.014532 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-afff3a8a-bfa6-4b37-b042-653eab2c24c3 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:16:41.015769 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-afff3a8a-bfa6-4b37-b042-653eab2c24c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:16:41.023974 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 774/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 22 19:16:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:16:41.567960 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aff59d-6f97-4ea9-a373-bec8c9f29448 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:16:41.569761 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-a4aff59d-6f97-4ea9-a373-bec8c9f29448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:16:41.633571 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:16:41.646682 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aff59d-6f97-4ea9-a373-bec8c9f29448 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:16:41.650840 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c8b3f4-c001-46c2-aa3a-b2c83087200e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e79498873d4ed29a0e19bbea9b9214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:16:41.651442 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c8b3f4-c001-46c2-aa3a-b2c83087200e service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:16:41.651746 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c8b3f4-c001-46c2-aa3a-b2c83087200e service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:16:41.652019 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c8b3f4-c001-46c2-aa3a-b2c83087200e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:16:41.660782 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c8b3f4-c001-46c2-aa3a-b2c83087200e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:16:41.662191 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c8b3f4-c001-46c2-aa3a-b2c83087200e service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:16:41.671470 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 762/1536] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:16:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:16:43.615745 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a87c7c1-8669-4566-b0e5-319b04d158f2 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:16:43.616370 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a87c7c1-8669-4566-b0e5-319b04d158f2 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:16:43.616370 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a87c7c1-8669-4566-b0e5-319b04d158f2 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:16:43.618231 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 775/1537] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 22 19:16:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:16:43.623444 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f76703-0c20-4be4-bcb9-a9399e1d7b5a None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:16:43.623929 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f76703-0c20-4be4-bcb9-a9399e1d7b5a None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:16:43.624038 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f76703-0c20-4be4-bcb9-a9399e1d7b5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:16:43.643140 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-27f76703-0c20-4be4-bcb9-a9399e1d7b5a None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:16:43.644403 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-27f76703-0c20-4be4-bcb9-a9399e1d7b5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:16:43.652476 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 763/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 22 19:16:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:16:44.204572 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a87c7c1-8669-4566-b0e5-319b04d158f2 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:16:44.206258 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-2a87c7c1-8669-4566-b0e5-319b04d158f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:16:44.219686 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a87c7c1-8669-4566-b0e5-319b04d158f2 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:16:44.223410 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92667c36-ff06-4cd8-a3ca-de3205e2904b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e79498873d4ed29a0e19bbea9b9214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:16:44.224046 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92667c36-ff06-4cd8-a3ca-de3205e2904b service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:16:44.224323 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92667c36-ff06-4cd8-a3ca-de3205e2904b service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:16:44.224593 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92667c36-ff06-4cd8-a3ca-de3205e2904b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:16:44.233820 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92667c36-ff06-4cd8-a3ca-de3205e2904b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:16:44.235239 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92667c36-ff06-4cd8-a3ca-de3205e2904b service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:16:44.245247 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 776/1539] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:16:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:16:53.638490 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=11 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:16:54.240537 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=14 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:18:06.679170 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff38c47-ed4f-4240-a99b-01ee3ebc1cd1 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:06.680371 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff38c47-ed4f-4240-a99b-01ee3ebc1cd1 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:06.680871 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff38c47-ed4f-4240-a99b-01ee3ebc1cd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:06.707100 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-eff38c47-ed4f-4240-a99b-01ee3ebc1cd1 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:18:06.712748 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-eff38c47-ed4f-4240-a99b-01ee3ebc1cd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:06.722793 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 764/1540] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:18:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:18:06.731226 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6924cd9-2ea0-4d34-92eb-5d72333adbd8 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:06.732244 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6924cd9-2ea0-4d34-92eb-5d72333adbd8 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:06.732734 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6924cd9-2ea0-4d34-92eb-5d72333adbd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:06.759452 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-b6924cd9-2ea0-4d34-92eb-5d72333adbd8 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:18:06.761704 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-b6924cd9-2ea0-4d34-92eb-5d72333adbd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:06.775398 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 777/1541] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:18:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:18:06.780374 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff38c47-ed4f-4240-a99b-01ee3ebc1cd1 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:06.801332 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff38c47-ed4f-4240-a99b-01ee3ebc1cd1 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:06.816771 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ccb1f7-f29f-4976-9cb5-6f9d926202b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:06.817811 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ccb1f7-f29f-4976-9cb5-6f9d926202b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:06.818304 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ccb1f7-f29f-4976-9cb5-6f9d926202b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:06.818807 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ccb1f7-f29f-4976-9cb5-6f9d926202b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:06.819890 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ccb1f7-f29f-4976-9cb5-6f9d926202b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:18:06.820411 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ccb1f7-f29f-4976-9cb5-6f9d926202b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:06.822235 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ccb1f7-f29f-4976-9cb5-6f9d926202b4 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:06.841172 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 765/1542] 10.222.0.61 () {58 vars in 1153 bytes} [Mon Aug 22 19:18:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:18:06.850097 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6924cd9-2ea0-4d34-92eb-5d72333adbd8 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:06.865606 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6924cd9-2ea0-4d34-92eb-5d72333adbd8 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:06.871805 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1417f631-3b1b-4a0e-9939-6840f01fb983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:06.872749 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1417f631-3b1b-4a0e-9939-6840f01fb983 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:06.873241 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1417f631-3b1b-4a0e-9939-6840f01fb983 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:06.873723 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1417f631-3b1b-4a0e-9939-6840f01fb983 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:06.874585 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1417f631-3b1b-4a0e-9939-6840f01fb983 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:06.876738 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1417f631-3b1b-4a0e-9939-6840f01fb983 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:06.929150 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 778/1543] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 19:18:06 2022] POST /identity/v3/projects => generated 371 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:18:06.934183 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ccb1f7-f29f-4976-9cb5-6f9d926202b4 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:06.945743 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ccb1f7-f29f-4976-9cb5-6f9d926202b4 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:07.036171 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842d9c85-02f2-4b05-98e6-2275d88e5e71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:07.037253 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842d9c85-02f2-4b05-98e6-2275d88e5e71 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:07.037758 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842d9c85-02f2-4b05-98e6-2275d88e5e71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:07.038250 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842d9c85-02f2-4b05-98e6-2275d88e5e71 admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:07.039120 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842d9c85-02f2-4b05-98e6-2275d88e5e71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:07.041564 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842d9c85-02f2-4b05-98e6-2275d88e5e71 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:07.084505 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 766/1544] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 19:18:06 2022] POST /identity/v3/users => generated 286 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 19:18:07.088756 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1417f631-3b1b-4a0e-9939-6840f01fb983 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:07.106676 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1417f631-3b1b-4a0e-9939-6840f01fb983 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:07.111832 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf60034-e54f-4d45-8e29-2ce68d5b51b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:07.112480 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf60034-e54f-4d45-8e29-2ce68d5b51b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:07.112765 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf60034-e54f-4d45-8e29-2ce68d5b51b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:07.113042 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf60034-e54f-4d45-8e29-2ce68d5b51b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:07.113709 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf60034-e54f-4d45-8e29-2ce68d5b51b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:07.119675 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf60034-e54f-4d45-8e29-2ce68d5b51b5 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:07.135263 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 779/1545] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:18:07 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:18:07.148070 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842d9c85-02f2-4b05-98e6-2275d88e5e71 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:07.172520 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842d9c85-02f2-4b05-98e6-2275d88e5e71 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:07.182849 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09f38d3-1721-4c8f-8729-0b529164caa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:07.184173 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09f38d3-1721-4c8f-8729-0b529164caa3 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:07.184567 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09f38d3-1721-4c8f-8729-0b529164caa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:07.184787 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09f38d3-1721-4c8f-8729-0b529164caa3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:07.185502 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09f38d3-1721-4c8f-8729-0b529164caa3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:07.187808 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09f38d3-1721-4c8f-8729-0b529164caa3 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:07.207272 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 767/1546] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:18:07 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:18:07.211684 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf60034-e54f-4d45-8e29-2ce68d5b51b5 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:07.229501 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf60034-e54f-4d45-8e29-2ce68d5b51b5 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:07.235095 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc89ab5b-5bb7-4083-a460-a9552048580e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:07.236006 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc89ab5b-5bb7-4083-a460-a9552048580e admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:07.236469 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc89ab5b-5bb7-4083-a460-a9552048580e admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:07.236917 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc89ab5b-5bb7-4083-a460-a9552048580e admin admin] PATH_INFO: `/v3/projects/64b20c01ca6b4ce380f8276431cd604d/users/a477d44f9f8646a3a123421659323775/roles/74a40bc819cb42eea7e966a4b2a016f1` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:07.245548 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839c7aeb-f91a-4090-9d63-215d63cdece3 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:07.245837 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839c7aeb-f91a-4090-9d63-215d63cdece3 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:07.246115 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839c7aeb-f91a-4090-9d63-215d63cdece3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:07.259188 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc89ab5b-5bb7-4083-a460-a9552048580e admin admin] RBAC: Authorizing `identity:create_grant(project_id=64b20c01ca6b4ce380f8276431cd604d, user_id=a477d44f9f8646a3a123421659323775, role_id=74a40bc819cb42eea7e966a4b2a016f1)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:07.262219 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc89ab5b-5bb7-4083-a460-a9552048580e admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:07.278532 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-839c7aeb-f91a-4090-9d63-215d63cdece3 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:18:07.280394 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-839c7aeb-f91a-4090-9d63-215d63cdece3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:07.289157 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 780/1547] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:18:07 2022] PUT /identity/v3/projects/64b20c01ca6b4ce380f8276431cd604d/users/a477d44f9f8646a3a123421659323775/roles/74a40bc819cb42eea7e966a4b2a016f1 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:18:07.296724 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d1db8a-5731-4f9e-ac04-de0880132cb1 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:07.297216 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d1db8a-5731-4f9e-ac04-de0880132cb1 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:07.297683 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d1db8a-5731-4f9e-ac04-de0880132cb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:07.325552 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-76d1db8a-5731-4f9e-ac04-de0880132cb1 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:18:07.327062 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-76d1db8a-5731-4f9e-ac04-de0880132cb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:07.401341 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 768/1548] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:18:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:18:07.404398 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bed1554-36bc-49bf-8619-e0ee075917d3 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:07.404685 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bed1554-36bc-49bf-8619-e0ee075917d3 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:07.404960 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bed1554-36bc-49bf-8619-e0ee075917d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:07.408072 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 781/1549] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:18:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 19:18:07.409426 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cbe416-7109-4ae2-a85c-0e41ff4d74ca None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:07.409713 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cbe416-7109-4ae2-a85c-0e41ff4d74ca None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:07.409988 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cbe416-7109-4ae2-a85c-0e41ff4d74ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:07.441461 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-66cbe416-7109-4ae2-a85c-0e41ff4d74ca None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:18:07.442946 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-66cbe416-7109-4ae2-a85c-0e41ff4d74ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:07.451615 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-5bed1554-36bc-49bf-8619-e0ee075917d3 None None] MFA Rules not processed for user `a477d44f9f8646a3a123421659323775`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:18:07.452509 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 782/1550] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:18:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:18:07.453379 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-5bed1554-36bc-49bf-8619-e0ee075917d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:07.453739 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556aadcb-5176-4729-8fc2-8a9491da8409 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:07.454021 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556aadcb-5176-4729-8fc2-8a9491da8409 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:07.454426 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556aadcb-5176-4729-8fc2-8a9491da8409 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:07.474191 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-556aadcb-5176-4729-8fc2-8a9491da8409 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:18:07.475538 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-556aadcb-5176-4729-8fc2-8a9491da8409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:07.483810 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 783/1551] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:18:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:18:07.485110 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad07106-bdc1-4d96-8acb-16d0395be6ff None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:07.485394 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad07106-bdc1-4d96-8acb-16d0395be6ff None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:07.485666 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad07106-bdc1-4d96-8acb-16d0395be6ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:07.505799 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-dad07106-bdc1-4d96-8acb-16d0395be6ff None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:18:07.506991 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-dad07106-bdc1-4d96-8acb-16d0395be6ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:07.507740 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:18:07.516072 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 784/1552] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:18:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:18:07.517263 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7306fb3e-1b22-439e-8cd7-4b8046d08eb7 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:07.517542 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7306fb3e-1b22-439e-8cd7-4b8046d08eb7 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:07.517812 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7306fb3e-1b22-439e-8cd7-4b8046d08eb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:07.531930 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 769/1553] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:18:07 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:18:07.533163 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d0bbdb-bd35-4a63-96cc-ebda2f61f321 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:07.533445 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d0bbdb-bd35-4a63-96cc-ebda2f61f321 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:07.533712 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d0bbdb-bd35-4a63-96cc-ebda2f61f321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:07.536571 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-7306fb3e-1b22-439e-8cd7-4b8046d08eb7 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:18:07.537789 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-7306fb3e-1b22-439e-8cd7-4b8046d08eb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:07.546039 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 785/1554] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:18:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:18:07.547227 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc17d4ae-e4cd-49f2-b174-4c89fbeb4c64 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:07.547657 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc17d4ae-e4cd-49f2-b174-4c89fbeb4c64 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:07.547948 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc17d4ae-e4cd-49f2-b174-4c89fbeb4c64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:07.552591 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-00d0bbdb-bd35-4a63-96cc-ebda2f61f321 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:18:07.553793 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-00d0bbdb-bd35-4a63-96cc-ebda2f61f321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:07.562329 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 770/1555] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:18:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:18:07.563669 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b95e246-a95d-4f6a-b2a5-dc27b0326ab1 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:07.563953 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b95e246-a95d-4f6a-b2a5-dc27b0326ab1 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:07.564245 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b95e246-a95d-4f6a-b2a5-dc27b0326ab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:07.571514 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-dc17d4ae-e4cd-49f2-b174-4c89fbeb4c64 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:18:07.572985 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-dc17d4ae-e4cd-49f2-b174-4c89fbeb4c64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:07.583041 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 786/1556] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:18:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:18:07.584341 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8381679c-77c4-4da7-85e3-286acb1258a6 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:07.584620 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8381679c-77c4-4da7-85e3-286acb1258a6 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:07.584887 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8381679c-77c4-4da7-85e3-286acb1258a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:07.585988 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-1b95e246-a95d-4f6a-b2a5-dc27b0326ab1 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:18:07.587197 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-1b95e246-a95d-4f6a-b2a5-dc27b0326ab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:07.596852 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 771/1557] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:18:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:18:07.597594 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972bff61-df58-410f-b323-3aa690039700 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:07.597878 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972bff61-df58-410f-b323-3aa690039700 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:07.598147 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972bff61-df58-410f-b323-3aa690039700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:07.607631 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-8381679c-77c4-4da7-85e3-286acb1258a6 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:18:07.608860 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-8381679c-77c4-4da7-85e3-286acb1258a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:07.616831 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 787/1558] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:18:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:18:07.617738 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8381679c-77c4-4da7-85e3-286acb1258a6 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:07.618277 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-972bff61-df58-410f-b323-3aa690039700 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:18:07.619444 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-972bff61-df58-410f-b323-3aa690039700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:07.627864 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8381679c-77c4-4da7-85e3-286acb1258a6 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:07.628275 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 772/1559] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:18:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:18:07.629033 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972bff61-df58-410f-b323-3aa690039700 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:07.633288 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad00c37-8b3b-4d14-a79c-e6f282009792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:07.633891 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad00c37-8b3b-4d14-a79c-e6f282009792 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:07.634168 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad00c37-8b3b-4d14-a79c-e6f282009792 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:07.634437 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad00c37-8b3b-4d14-a79c-e6f282009792 admin admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:07.635117 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad00c37-8b3b-4d14-a79c-e6f282009792 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:18:07.635419 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad00c37-8b3b-4d14-a79c-e6f282009792 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:07.636510 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: WARNING py.warnings [None req-1ad00c37-8b3b-4d14-a79c-e6f282009792 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 22 19:18:07.636510 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: warnings.warn(msg) Aug 22 19:18:07.636510 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: Aug 22 19:18:07.637399 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad00c37-8b3b-4d14-a79c-e6f282009792 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:07.638592 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972bff61-df58-410f-b323-3aa690039700 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:07.643164 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7123a3d8-b981-435f-bfb3-76195f3e269f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:07.643777 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7123a3d8-b981-435f-bfb3-76195f3e269f admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:07.644059 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7123a3d8-b981-435f-bfb3-76195f3e269f admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:07.644332 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7123a3d8-b981-435f-bfb3-76195f3e269f admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:07.644953 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7123a3d8-b981-435f-bfb3-76195f3e269f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:07.645951 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: WARNING py.warnings [None req-7123a3d8-b981-435f-bfb3-76195f3e269f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 22 19:18:07.645951 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: warnings.warn(msg) Aug 22 19:18:07.645951 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: Aug 22 19:18:07.647125 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7123a3d8-b981-435f-bfb3-76195f3e269f admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:07.649178 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 788/1560] 10.222.0.61 () {58 vars in 1153 bytes} [Mon Aug 22 19:18:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:18:07.649870 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad00c37-8b3b-4d14-a79c-e6f282009792 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:07.660719 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad00c37-8b3b-4d14-a79c-e6f282009792 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:07.665865 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235e79df-3ed9-4865-bcb1-79c6bbf0724c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:07.666455 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235e79df-3ed9-4865-bcb1-79c6bbf0724c admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:07.666787 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235e79df-3ed9-4865-bcb1-79c6bbf0724c admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:07.667061 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235e79df-3ed9-4865-bcb1-79c6bbf0724c admin admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:07.668313 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235e79df-3ed9-4865-bcb1-79c6bbf0724c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:18:07.668678 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235e79df-3ed9-4865-bcb1-79c6bbf0724c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:07.670369 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235e79df-3ed9-4865-bcb1-79c6bbf0724c admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:07.675851 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 773/1561] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 19:18:07 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:18:07.677122 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a68f05e-c4c6-47e5-a058-99a50297615b None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:07.677402 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a68f05e-c4c6-47e5-a058-99a50297615b None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:07.677672 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a68f05e-c4c6-47e5-a058-99a50297615b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:07.691828 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 789/1562] 10.222.0.61 () {58 vars in 1153 bytes} [Mon Aug 22 19:18:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:18:07.691828 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2c9e9d-a827-4217-809c-9dc3d46e7111 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:07.691828 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2c9e9d-a827-4217-809c-9dc3d46e7111 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:07.691828 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2c9e9d-a827-4217-809c-9dc3d46e7111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:07.704647 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-2a68f05e-c4c6-47e5-a058-99a50297615b None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:18:07.705179 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-2e2c9e9d-a827-4217-809c-9dc3d46e7111 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:18:07.705885 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-2a68f05e-c4c6-47e5-a058-99a50297615b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:07.706340 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-2e2c9e9d-a827-4217-809c-9dc3d46e7111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:07.725435 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:18:07.768863 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 774/1563] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:18:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:18:07.769784 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a68f05e-c4c6-47e5-a058-99a50297615b None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:07.774132 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 790/1564] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:18:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:18:07.776138 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f6a891-4f0f-448d-9664-611ecf8608ab None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:07.776455 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f6a891-4f0f-448d-9664-611ecf8608ab None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:07.776759 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f6a891-4f0f-448d-9664-611ecf8608ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:07.780115 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a68f05e-c4c6-47e5-a058-99a50297615b None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:07.785741 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e863408-aeaf-4db1-adf9-e7bbd875dcfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:07.786352 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e863408-aeaf-4db1-adf9-e7bbd875dcfa admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:07.786633 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e863408-aeaf-4db1-adf9-e7bbd875dcfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:07.786909 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e863408-aeaf-4db1-adf9-e7bbd875dcfa admin admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:07.787562 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e863408-aeaf-4db1-adf9-e7bbd875dcfa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:18:07.787965 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e863408-aeaf-4db1-adf9-e7bbd875dcfa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:07.789553 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e863408-aeaf-4db1-adf9-e7bbd875dcfa admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:07.796021 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-e3f6a891-4f0f-448d-9664-611ecf8608ab None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:18:07.797215 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-e3f6a891-4f0f-448d-9664-611ecf8608ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:07.801924 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 775/1565] 10.222.0.61 () {58 vars in 1153 bytes} [Mon Aug 22 19:18:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:18:07.802726 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e863408-aeaf-4db1-adf9-e7bbd875dcfa admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:07.804967 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 791/1566] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:18:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:18:07.805707 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f6a891-4f0f-448d-9664-611ecf8608ab None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:07.818649 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e863408-aeaf-4db1-adf9-e7bbd875dcfa admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:07.819124 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f6a891-4f0f-448d-9664-611ecf8608ab None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:07.823178 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50574884-3c6b-42a5-a57a-80e091c77aed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:07.823809 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d7bc2c-b748-4f5f-b43d-a55c83088ee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:07.823958 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50574884-3c6b-42a5-a57a-80e091c77aed admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:07.824074 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50574884-3c6b-42a5-a57a-80e091c77aed admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:07.824347 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50574884-3c6b-42a5-a57a-80e091c77aed admin admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:07.824532 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d7bc2c-b748-4f5f-b43d-a55c83088ee8 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:07.824792 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d7bc2c-b748-4f5f-b43d-a55c83088ee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:07.824989 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50574884-3c6b-42a5-a57a-80e091c77aed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:18:07.825124 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d7bc2c-b748-4f5f-b43d-a55c83088ee8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:07.825311 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50574884-3c6b-42a5-a57a-80e091c77aed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:07.825718 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d7bc2c-b748-4f5f-b43d-a55c83088ee8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:07.826863 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50574884-3c6b-42a5-a57a-80e091c77aed admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:07.827715 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d7bc2c-b748-4f5f-b43d-a55c83088ee8 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:07.838710 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 776/1567] 10.222.0.61 () {58 vars in 1153 bytes} [Mon Aug 22 19:18:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:18:07.839491 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50574884-3c6b-42a5-a57a-80e091c77aed admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:07.854345 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50574884-3c6b-42a5-a57a-80e091c77aed admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:07.855810 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 792/1568] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 19:18:07 2022] POST /identity/v3/projects => generated 375 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:18:07.856677 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d7bc2c-b748-4f5f-b43d-a55c83088ee8 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:07.914577 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9416954c-136b-40ff-a34b-aa15b2c9a503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:07.915191 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9416954c-136b-40ff-a34b-aa15b2c9a503 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:07.915475 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9416954c-136b-40ff-a34b-aa15b2c9a503 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:07.915776 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9416954c-136b-40ff-a34b-aa15b2c9a503 admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:07.916396 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9416954c-136b-40ff-a34b-aa15b2c9a503 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:07.918847 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9416954c-136b-40ff-a34b-aa15b2c9a503 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:07.919139 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d7bc2c-b748-4f5f-b43d-a55c83088ee8 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:07.923722 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c100a4-6994-4204-a325-ccb7c18801d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:07.924325 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c100a4-6994-4204-a325-ccb7c18801d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:07.924600 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c100a4-6994-4204-a325-ccb7c18801d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:07.924867 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c100a4-6994-4204-a325-ccb7c18801d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:07.925478 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c100a4-6994-4204-a325-ccb7c18801d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:07.927357 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c100a4-6994-4204-a325-ccb7c18801d5 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:07.940498 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 777/1569] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 19:18:07 2022] POST /identity/v3/users => generated 293 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:18:07.941515 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9416954c-136b-40ff-a34b-aa15b2c9a503 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:07.951489 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9416954c-136b-40ff-a34b-aa15b2c9a503 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:07.956012 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184555c2-b029-4eba-98de-6f17c0fcd8cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:07.956595 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184555c2-b029-4eba-98de-6f17c0fcd8cd admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:07.956873 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184555c2-b029-4eba-98de-6f17c0fcd8cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:07.957144 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184555c2-b029-4eba-98de-6f17c0fcd8cd admin admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:07.957783 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184555c2-b029-4eba-98de-6f17c0fcd8cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:18:07.958112 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184555c2-b029-4eba-98de-6f17c0fcd8cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:07.958306 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 793/1570] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 19:18:07 2022] POST /identity/v3/projects => generated 365 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:18:07.959122 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c100a4-6994-4204-a325-ccb7c18801d5 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:07.959727 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184555c2-b029-4eba-98de-6f17c0fcd8cd admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:07.967231 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:18:07.973799 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 778/1571] 10.222.0.61 () {58 vars in 1153 bytes} [Mon Aug 22 19:18:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:18:07.975248 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184555c2-b029-4eba-98de-6f17c0fcd8cd admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:07.986263 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184555c2-b029-4eba-98de-6f17c0fcd8cd admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:08.028580 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c100a4-6994-4204-a325-ccb7c18801d5 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:08.034656 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11f51ec-e025-47da-a9e7-8466d2aea6b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:08.035264 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11f51ec-e025-47da-a9e7-8466d2aea6b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:08.035540 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11f51ec-e025-47da-a9e7-8466d2aea6b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:08.035839 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11f51ec-e025-47da-a9e7-8466d2aea6b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:08.036480 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11f51ec-e025-47da-a9e7-8466d2aea6b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:18:08.036780 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11f51ec-e025-47da-a9e7-8466d2aea6b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:08.038434 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11f51ec-e025-47da-a9e7-8466d2aea6b4 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:08.045186 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10298789-b6c4-46be-869d-8335ec2e51ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:08.045806 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10298789-b6c4-46be-869d-8335ec2e51ce admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:08.046086 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10298789-b6c4-46be-869d-8335ec2e51ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:08.046358 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10298789-b6c4-46be-869d-8335ec2e51ce admin admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:08.047005 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10298789-b6c4-46be-869d-8335ec2e51ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:18:08.047307 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10298789-b6c4-46be-869d-8335ec2e51ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:08.049074 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10298789-b6c4-46be-869d-8335ec2e51ce admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:08.050891 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 794/1572] 10.222.0.61 () {58 vars in 1153 bytes} [Mon Aug 22 19:18:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:18:08.051841 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11f51ec-e025-47da-a9e7-8466d2aea6b4 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:08.062011 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11f51ec-e025-47da-a9e7-8466d2aea6b4 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:08.062559 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 779/1573] 10.222.0.61 () {58 vars in 1153 bytes} [Mon Aug 22 19:18:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:18:08.063361 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10298789-b6c4-46be-869d-8335ec2e51ce admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:08.066990 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4932549-04e6-46c6-a3be-fb15e00b4349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:08.067754 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4932549-04e6-46c6-a3be-fb15e00b4349 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:08.068042 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4932549-04e6-46c6-a3be-fb15e00b4349 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:08.068462 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4932549-04e6-46c6-a3be-fb15e00b4349 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:08.069096 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4932549-04e6-46c6-a3be-fb15e00b4349 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:08.071099 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4932549-04e6-46c6-a3be-fb15e00b4349 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:08.074417 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10298789-b6c4-46be-869d-8335ec2e51ce admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:08.079027 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfef00aa-0c19-4ad0-9e00-f764f1830ad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:08.079612 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfef00aa-0c19-4ad0-9e00-f764f1830ad4 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:08.079928 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfef00aa-0c19-4ad0-9e00-f764f1830ad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:08.080197 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfef00aa-0c19-4ad0-9e00-f764f1830ad4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:08.080987 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfef00aa-0c19-4ad0-9e00-f764f1830ad4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:08.082693 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfef00aa-0c19-4ad0-9e00-f764f1830ad4 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:08.082879 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:18:08.102030 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 795/1574] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 19:18:08 2022] POST /identity/v3/projects => generated 379 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:18:08.102961 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4932549-04e6-46c6-a3be-fb15e00b4349 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:08.114196 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 780/1575] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 19:18:08 2022] POST /identity/v3/projects => generated 405 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:18:08.115030 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfef00aa-0c19-4ad0-9e00-f764f1830ad4 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:08.173724 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4932549-04e6-46c6-a3be-fb15e00b4349 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:08.177280 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfef00aa-0c19-4ad0-9e00-f764f1830ad4 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:08.178380 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edbe530-3bca-4c3c-b311-082b4b71e134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:08.178988 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edbe530-3bca-4c3c-b311-082b4b71e134 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:08.179259 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edbe530-3bca-4c3c-b311-082b4b71e134 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:08.179528 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edbe530-3bca-4c3c-b311-082b4b71e134 admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:08.180170 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edbe530-3bca-4c3c-b311-082b4b71e134 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:08.181174 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: WARNING py.warnings [None req-5edbe530-3bca-4c3c-b311-082b4b71e134 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 22 19:18:08.181174 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: warnings.warn(msg) Aug 22 19:18:08.181174 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: Aug 22 19:18:08.181905 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70476eb4-5bb3-466a-a477-83976419f1d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:08.182365 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edbe530-3bca-4c3c-b311-082b4b71e134 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:08.182510 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70476eb4-5bb3-466a-a477-83976419f1d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:08.182794 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70476eb4-5bb3-466a-a477-83976419f1d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:08.183119 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70476eb4-5bb3-466a-a477-83976419f1d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:08.183871 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70476eb4-5bb3-466a-a477-83976419f1d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:08.185464 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70476eb4-5bb3-466a-a477-83976419f1d8 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:08.195598 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 781/1576] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:18:08 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:18:08.196431 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70476eb4-5bb3-466a-a477-83976419f1d8 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:08.206111 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 796/1577] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 19:18:08 2022] POST /identity/v3/users => generated 288 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:18:08.206826 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edbe530-3bca-4c3c-b311-082b4b71e134 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:08.209733 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70476eb4-5bb3-466a-a477-83976419f1d8 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:08.215073 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb06c56-cef0-46b4-a292-1fd7a8f5e5c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:08.215854 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb06c56-cef0-46b4-a292-1fd7a8f5e5c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:08.216231 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb06c56-cef0-46b4-a292-1fd7a8f5e5c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:08.216600 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb06c56-cef0-46b4-a292-1fd7a8f5e5c8 admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:08.216707 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edbe530-3bca-4c3c-b311-082b4b71e134 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:08.217400 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb06c56-cef0-46b4-a292-1fd7a8f5e5c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:08.219448 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb06c56-cef0-46b4-a292-1fd7a8f5e5c8 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:08.221483 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e2259d-be2b-4edb-8b63-e09d3b968509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:08.222057 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e2259d-be2b-4edb-8b63-e09d3b968509 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:08.222330 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e2259d-be2b-4edb-8b63-e09d3b968509 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:08.222598 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e2259d-be2b-4edb-8b63-e09d3b968509 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:08.223204 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e2259d-be2b-4edb-8b63-e09d3b968509 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:08.225088 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e2259d-be2b-4edb-8b63-e09d3b968509 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:08.245621 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 782/1578] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 19:18:08 2022] POST /identity/v3/users => generated 283 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:18:08.245621 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb06c56-cef0-46b4-a292-1fd7a8f5e5c8 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:08.253230 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb06c56-cef0-46b4-a292-1fd7a8f5e5c8 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:08.256713 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 797/1579] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 19:18:08 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:18:08.257356 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e2259d-be2b-4edb-8b63-e09d3b968509 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:08.261025 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8812906-f5a9-440e-a20e-8d15d49d0f6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:08.261641 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8812906-f5a9-440e-a20e-8d15d49d0f6f admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:08.261920 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8812906-f5a9-440e-a20e-8d15d49d0f6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:08.262188 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8812906-f5a9-440e-a20e-8d15d49d0f6f admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:08.262798 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8812906-f5a9-440e-a20e-8d15d49d0f6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:08.264757 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8812906-f5a9-440e-a20e-8d15d49d0f6f admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:08.291984 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 783/1580] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 19:18:08 2022] POST /identity/v3/projects => generated 393 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:18:08.293516 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8812906-f5a9-440e-a20e-8d15d49d0f6f admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:08.322466 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:18:08.325154 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e2259d-be2b-4edb-8b63-e09d3b968509 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:08.358901 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8812906-f5a9-440e-a20e-8d15d49d0f6f admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:08.363907 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68f98a8-754b-43a1-9324-9588b984ca1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:08.364516 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68f98a8-754b-43a1-9324-9588b984ca1f admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:08.364875 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68f98a8-754b-43a1-9324-9588b984ca1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:08.365151 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68f98a8-754b-43a1-9324-9588b984ca1f admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:08.365803 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68f98a8-754b-43a1-9324-9588b984ca1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:08.367947 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68f98a8-754b-43a1-9324-9588b984ca1f admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:08.382817 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faed1b92-a4fb-4ac0-b75d-bc7247af9583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:08.383421 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faed1b92-a4fb-4ac0-b75d-bc7247af9583 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:08.383721 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faed1b92-a4fb-4ac0-b75d-bc7247af9583 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:08.383993 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faed1b92-a4fb-4ac0-b75d-bc7247af9583 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:08.384607 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faed1b92-a4fb-4ac0-b75d-bc7247af9583 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:08.387028 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faed1b92-a4fb-4ac0-b75d-bc7247af9583 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:08.388480 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 784/1581] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 19:18:08 2022] POST /identity/v3/users => generated 290 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:18:08.389266 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68f98a8-754b-43a1-9324-9588b984ca1f admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:08.398440 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:18:08.398593 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68f98a8-754b-43a1-9324-9588b984ca1f admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:08.404121 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae15f1c9-d511-473e-bcff-4816b8d2e26c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:08.404717 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae15f1c9-d511-473e-bcff-4816b8d2e26c admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:08.404996 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae15f1c9-d511-473e-bcff-4816b8d2e26c admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:08.405307 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae15f1c9-d511-473e-bcff-4816b8d2e26c admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:08.405884 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae15f1c9-d511-473e-bcff-4816b8d2e26c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:08.407805 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae15f1c9-d511-473e-bcff-4816b8d2e26c admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:08.415725 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 798/1582] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 19:18:08 2022] POST /identity/v3/projects => generated 379 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:18:08.419779 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faed1b92-a4fb-4ac0-b75d-bc7247af9583 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:08.430153 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 785/1583] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 19:18:08 2022] POST /identity/v3/users => generated 303 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:18:08.431247 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae15f1c9-d511-473e-bcff-4816b8d2e26c admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:08.440724 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae15f1c9-d511-473e-bcff-4816b8d2e26c admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:08.483946 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faed1b92-a4fb-4ac0-b75d-bc7247af9583 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:08.488543 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78df911-3753-4911-a604-953f03b5a876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:08.489200 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78df911-3753-4911-a604-953f03b5a876 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:08.489477 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78df911-3753-4911-a604-953f03b5a876 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:08.489746 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78df911-3753-4911-a604-953f03b5a876 admin admin] PATH_INFO: `/v3/projects/3b4f0102ce5749fe82b66a3943b0df76/users/570f8f1038ca474bba31f467e74a17d0/roles/163c30c3b3c246beb849ce176f02d9da` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:08.499578 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ea3c1f-5f44-40d2-9270-86285e27c2aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:08.500237 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ea3c1f-5f44-40d2-9270-86285e27c2aa admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:08.500521 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ea3c1f-5f44-40d2-9270-86285e27c2aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:08.500797 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ea3c1f-5f44-40d2-9270-86285e27c2aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:08.501439 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ea3c1f-5f44-40d2-9270-86285e27c2aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:08.503039 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ea3c1f-5f44-40d2-9270-86285e27c2aa admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:08.503984 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78df911-3753-4911-a604-953f03b5a876 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b4f0102ce5749fe82b66a3943b0df76, user_id=570f8f1038ca474bba31f467e74a17d0, role_id=163c30c3b3c246beb849ce176f02d9da)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:08.506615 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78df911-3753-4911-a604-953f03b5a876 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:08.514230 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 786/1584] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:18:08 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:18:08.515490 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ea3c1f-5f44-40d2-9270-86285e27c2aa admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:08.520367 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 799/1585] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:18:08 2022] PUT /identity/v3/projects/3b4f0102ce5749fe82b66a3943b0df76/users/570f8f1038ca474bba31f467e74a17d0/roles/163c30c3b3c246beb849ce176f02d9da => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:18:08.524344 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78df911-3753-4911-a604-953f03b5a876 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:08.525658 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ea3c1f-5f44-40d2-9270-86285e27c2aa admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:08.586313 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78df911-3753-4911-a604-953f03b5a876 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:08.588578 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aad4e21-2ad8-47d9-8f5e-bcb9fed675cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:08.589216 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aad4e21-2ad8-47d9-8f5e-bcb9fed675cf admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:08.590074 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aad4e21-2ad8-47d9-8f5e-bcb9fed675cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:08.590353 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aad4e21-2ad8-47d9-8f5e-bcb9fed675cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:08.590996 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aad4e21-2ad8-47d9-8f5e-bcb9fed675cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:08.591442 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ecef8e-e840-4497-ac52-5770133fa1f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:08.592077 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ecef8e-e840-4497-ac52-5770133fa1f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:08.592349 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ecef8e-e840-4497-ac52-5770133fa1f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:08.592612 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aad4e21-2ad8-47d9-8f5e-bcb9fed675cf admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:08.592718 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ecef8e-e840-4497-ac52-5770133fa1f3 admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:08.593224 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ecef8e-e840-4497-ac52-5770133fa1f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:08.595118 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ecef8e-e840-4497-ac52-5770133fa1f3 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:08.602735 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 787/1586] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:18:08 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:18:08.603350 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aad4e21-2ad8-47d9-8f5e-bcb9fed675cf admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:08.614681 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aad4e21-2ad8-47d9-8f5e-bcb9fed675cf admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:08.616986 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 800/1587] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 19:18:08 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:18:08.617850 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ecef8e-e840-4497-ac52-5770133fa1f3 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:08.622421 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b276642f-57fb-44c7-b68c-7b69f4ccf878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:08.623083 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b276642f-57fb-44c7-b68c-7b69f4ccf878 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:08.623370 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b276642f-57fb-44c7-b68c-7b69f4ccf878 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:08.623667 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b276642f-57fb-44c7-b68c-7b69f4ccf878 admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:08.624301 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b276642f-57fb-44c7-b68c-7b69f4ccf878 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:08.626314 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b276642f-57fb-44c7-b68c-7b69f4ccf878 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:08.627598 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ecef8e-e840-4497-ac52-5770133fa1f3 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:08.632118 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4f0117-6947-41d9-9ff4-d99fea76480c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:08.632691 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4f0117-6947-41d9-9ff4-d99fea76480c admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:08.632965 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4f0117-6947-41d9-9ff4-d99fea76480c admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:08.633235 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4f0117-6947-41d9-9ff4-d99fea76480c admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:08.633862 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4f0117-6947-41d9-9ff4-d99fea76480c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:08.635547 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4f0117-6947-41d9-9ff4-d99fea76480c admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:08.647140 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 801/1588] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:18:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:18:08.647940 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4f0117-6947-41d9-9ff4-d99fea76480c admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:08.652878 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 788/1589] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 19:18:08 2022] POST /identity/v3/users => generated 297 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:18:08.653750 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b276642f-57fb-44c7-b68c-7b69f4ccf878 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:08.658175 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4f0117-6947-41d9-9ff4-d99fea76480c admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:08.663004 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0ebc7b-7527-408e-b9ca-7184b9dbde0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:08.663589 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0ebc7b-7527-408e-b9ca-7184b9dbde0a admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:08.663905 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0ebc7b-7527-408e-b9ca-7184b9dbde0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:08.664154 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b276642f-57fb-44c7-b68c-7b69f4ccf878 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:08.664320 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0ebc7b-7527-408e-b9ca-7184b9dbde0a admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:08.664778 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0ebc7b-7527-408e-b9ca-7184b9dbde0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:08.666700 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0ebc7b-7527-408e-b9ca-7184b9dbde0a admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:08.669309 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5b4051-e3c0-495f-bc08-c76da63b8585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:08.669890 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5b4051-e3c0-495f-bc08-c76da63b8585 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:08.670163 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5b4051-e3c0-495f-bc08-c76da63b8585 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:08.670434 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5b4051-e3c0-495f-bc08-c76da63b8585 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:08.671068 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5b4051-e3c0-495f-bc08-c76da63b8585 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:08.672657 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5b4051-e3c0-495f-bc08-c76da63b8585 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:08.685316 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 789/1590] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:18:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:18:08.686141 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5b4051-e3c0-495f-bc08-c76da63b8585 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:08.690731 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 802/1591] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 19:18:08 2022] POST /identity/v3/users => generated 290 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:18:08.691466 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0ebc7b-7527-408e-b9ca-7184b9dbde0a admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:08.692637 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:18:08.698289 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5b4051-e3c0-495f-bc08-c76da63b8585 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:08.702800 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0ebc7b-7527-408e-b9ca-7184b9dbde0a admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:08.705231 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7a2bad-87f3-42d9-9d8c-e9b8cdd4d34b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:08.705859 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7a2bad-87f3-42d9-9d8c-e9b8cdd4d34b admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:08.706137 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7a2bad-87f3-42d9-9d8c-e9b8cdd4d34b admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:08.706399 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7a2bad-87f3-42d9-9d8c-e9b8cdd4d34b admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:08.707030 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7a2bad-87f3-42d9-9d8c-e9b8cdd4d34b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:08.707566 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09615e7-3325-404a-a015-180ebe415cc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:08.708327 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09615e7-3325-404a-a015-180ebe415cc7 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:08.708611 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09615e7-3325-404a-a015-180ebe415cc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:08.708719 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7a2bad-87f3-42d9-9d8c-e9b8cdd4d34b admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:08.708883 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09615e7-3325-404a-a015-180ebe415cc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:08.709521 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09615e7-3325-404a-a015-180ebe415cc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:08.711099 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09615e7-3325-404a-a015-180ebe415cc7 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:08.719178 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 790/1592] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:18:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:18:08.720005 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7a2bad-87f3-42d9-9d8c-e9b8cdd4d34b admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:08.724785 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 803/1593] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:18:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:18:08.724785 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09615e7-3325-404a-a015-180ebe415cc7 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:08.733013 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7a2bad-87f3-42d9-9d8c-e9b8cdd4d34b admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:08.733587 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09615e7-3325-404a-a015-180ebe415cc7 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:08.737625 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b53c0a-9ae3-43b2-8d8b-16f93cc3de62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:08.738147 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b53c0a-9ae3-43b2-8d8b-16f93cc3de62 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:08.738391 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1e5818-86af-4f70-8787-3d41c3c68590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:08.738508 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b53c0a-9ae3-43b2-8d8b-16f93cc3de62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:08.738671 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b53c0a-9ae3-43b2-8d8b-16f93cc3de62 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:08.738977 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1e5818-86af-4f70-8787-3d41c3c68590 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:08.739282 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1e5818-86af-4f70-8787-3d41c3c68590 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:08.739463 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b53c0a-9ae3-43b2-8d8b-16f93cc3de62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:08.739574 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1e5818-86af-4f70-8787-3d41c3c68590 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:08.740227 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1e5818-86af-4f70-8787-3d41c3c68590 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:08.740885 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b53c0a-9ae3-43b2-8d8b-16f93cc3de62 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:08.741798 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1e5818-86af-4f70-8787-3d41c3c68590 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:08.750795 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 791/1594] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:18:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:18:08.751825 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b53c0a-9ae3-43b2-8d8b-16f93cc3de62 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:08.751960 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 804/1595] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:18:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:18:08.752468 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1e5818-86af-4f70-8787-3d41c3c68590 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:08.761998 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1e5818-86af-4f70-8787-3d41c3c68590 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:08.765582 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b53c0a-9ae3-43b2-8d8b-16f93cc3de62 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:08.766559 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32fd05b-9f74-4635-982b-e90a18e3704b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:08.767134 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32fd05b-9f74-4635-982b-e90a18e3704b admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:08.767410 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32fd05b-9f74-4635-982b-e90a18e3704b admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:08.767701 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32fd05b-9f74-4635-982b-e90a18e3704b admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:08.768328 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32fd05b-9f74-4635-982b-e90a18e3704b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:08.769907 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32fd05b-9f74-4635-982b-e90a18e3704b admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:08.770087 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e0d9a9-d512-4641-a79e-90d6b87fb4d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:08.770693 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e0d9a9-d512-4641-a79e-90d6b87fb4d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:08.770970 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e0d9a9-d512-4641-a79e-90d6b87fb4d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:08.771235 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e0d9a9-d512-4641-a79e-90d6b87fb4d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:08.771886 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e0d9a9-d512-4641-a79e-90d6b87fb4d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:08.773430 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e0d9a9-d512-4641-a79e-90d6b87fb4d1 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:08.781066 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 805/1596] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:18:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:18:08.781937 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32fd05b-9f74-4635-982b-e90a18e3704b admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:08.784805 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 792/1597] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:18:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:18:08.785588 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e0d9a9-d512-4641-a79e-90d6b87fb4d1 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:08.791773 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:18:08.795715 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e0d9a9-d512-4641-a79e-90d6b87fb4d1 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:08.797760 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32fd05b-9f74-4635-982b-e90a18e3704b admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:08.800505 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb840c49-185d-4771-b166-9b1adf7d6a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:08.801079 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb840c49-185d-4771-b166-9b1adf7d6a0c admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:08.801356 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb840c49-185d-4771-b166-9b1adf7d6a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:08.801628 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb840c49-185d-4771-b166-9b1adf7d6a0c admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:08.802264 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb840c49-185d-4771-b166-9b1adf7d6a0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:08.803707 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bcd86c-3dc2-4bcf-a2c0-34058661dfe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:08.803869 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb840c49-185d-4771-b166-9b1adf7d6a0c admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:08.804325 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bcd86c-3dc2-4bcf-a2c0-34058661dfe6 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:08.804602 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bcd86c-3dc2-4bcf-a2c0-34058661dfe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:08.804869 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bcd86c-3dc2-4bcf-a2c0-34058661dfe6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:08.805534 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bcd86c-3dc2-4bcf-a2c0-34058661dfe6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:08.807152 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bcd86c-3dc2-4bcf-a2c0-34058661dfe6 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:08.813784 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 793/1598] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:18:08 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:18:08.813784 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb840c49-185d-4771-b166-9b1adf7d6a0c admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:08.820552 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 806/1599] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:18:08 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:18:08.820552 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bcd86c-3dc2-4bcf-a2c0-34058661dfe6 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:08.825673 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb840c49-185d-4771-b166-9b1adf7d6a0c admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:08.828029 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bcd86c-3dc2-4bcf-a2c0-34058661dfe6 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:08.830385 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925d9eca-58f1-4ec9-8062-66d975d3539e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:08.831034 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d9eca-58f1-4ec9-8062-66d975d3539e admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:08.831436 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d9eca-58f1-4ec9-8062-66d975d3539e admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:08.831756 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d9eca-58f1-4ec9-8062-66d975d3539e admin admin] PATH_INFO: `/v3/projects/ca75caa807b445f486899173277b5051/users/21d0ab6d1a724547b8c643287df897da/roles/74a40bc819cb42eea7e966a4b2a016f1` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:08.832513 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adefedd7-b474-4f03-abf0-7536e1605a79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:08.833148 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adefedd7-b474-4f03-abf0-7536e1605a79 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:08.833409 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adefedd7-b474-4f03-abf0-7536e1605a79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:08.833678 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adefedd7-b474-4f03-abf0-7536e1605a79 admin admin] PATH_INFO: `/v3/projects/3b4f0102ce5749fe82b66a3943b0df76/users/570f8f1038ca474bba31f467e74a17d0/roles/163c30c3b3c246beb849ce176f02d9da` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:08.835568 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adefedd7-b474-4f03-abf0-7536e1605a79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b4f0102ce5749fe82b66a3943b0df76, user_id=570f8f1038ca474bba31f467e74a17d0, role_id=163c30c3b3c246beb849ce176f02d9da)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:08.838205 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adefedd7-b474-4f03-abf0-7536e1605a79 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:08.846947 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925d9eca-58f1-4ec9-8062-66d975d3539e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca75caa807b445f486899173277b5051, user_id=21d0ab6d1a724547b8c643287df897da, role_id=74a40bc819cb42eea7e966a4b2a016f1)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:08.848157 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: WARNING py.warnings [None req-925d9eca-58f1-4ec9-8062-66d975d3539e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 22 19:18:08.848157 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: warnings.warn(msg) Aug 22 19:18:08.848157 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: Aug 22 19:18:08.849095 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 807/1600] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:18:08 2022] PUT /identity/v3/projects/3b4f0102ce5749fe82b66a3943b0df76/users/570f8f1038ca474bba31f467e74a17d0/roles/163c30c3b3c246beb849ce176f02d9da => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:18:08.849942 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adefedd7-b474-4f03-abf0-7536e1605a79 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:08.850115 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925d9eca-58f1-4ec9-8062-66d975d3539e admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:08.870313 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 794/1601] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:18:08 2022] PUT /identity/v3/projects/ca75caa807b445f486899173277b5051/users/21d0ab6d1a724547b8c643287df897da/roles/74a40bc819cb42eea7e966a4b2a016f1 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:18:08.871162 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925d9eca-58f1-4ec9-8062-66d975d3539e admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:08.914522 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adefedd7-b474-4f03-abf0-7536e1605a79 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:08.932976 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925d9eca-58f1-4ec9-8062-66d975d3539e admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:08.937523 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62fe30e-2480-4714-ae1e-326840c207fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:08.938123 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62fe30e-2480-4714-ae1e-326840c207fa admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:08.938396 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62fe30e-2480-4714-ae1e-326840c207fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:08.938666 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62fe30e-2480-4714-ae1e-326840c207fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:08.939306 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62fe30e-2480-4714-ae1e-326840c207fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:08.940931 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62fe30e-2480-4714-ae1e-326840c207fa admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:08.950431 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 795/1602] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:18:08 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:18:08.951211 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62fe30e-2480-4714-ae1e-326840c207fa admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:08.960460 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62fe30e-2480-4714-ae1e-326840c207fa admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:08.965551 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266151ad-eb1e-4234-b83c-640a692a924c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:08.966110 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266151ad-eb1e-4234-b83c-640a692a924c admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:08.966386 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266151ad-eb1e-4234-b83c-640a692a924c admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:08.966652 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266151ad-eb1e-4234-b83c-640a692a924c admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:08.967258 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266151ad-eb1e-4234-b83c-640a692a924c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:08.968847 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266151ad-eb1e-4234-b83c-640a692a924c admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:08.972424 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cf4584-7497-43c8-b6b8-796ebd9334fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:08.973079 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cf4584-7497-43c8-b6b8-796ebd9334fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:08.973353 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cf4584-7497-43c8-b6b8-796ebd9334fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:08.973625 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cf4584-7497-43c8-b6b8-796ebd9334fe admin admin] PATH_INFO: `/v3/projects/4c435019ac81431cb462873b6eaf636f/users/cdb4258fb5914636a40825b29c6e102e/roles/74a40bc819cb42eea7e966a4b2a016f1` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:08.978732 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 796/1603] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:18:08 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:18:08.979336 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266151ad-eb1e-4234-b83c-640a692a924c admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:08.988968 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cf4584-7497-43c8-b6b8-796ebd9334fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c435019ac81431cb462873b6eaf636f, user_id=cdb4258fb5914636a40825b29c6e102e, role_id=74a40bc819cb42eea7e966a4b2a016f1)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:08.989146 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266151ad-eb1e-4234-b83c-640a692a924c admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:08.991784 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cf4584-7497-43c8-b6b8-796ebd9334fe admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:08.993702 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2183ef-11e1-48f5-8cf2-66deae880185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:08.994324 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2183ef-11e1-48f5-8cf2-66deae880185 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:08.994598 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2183ef-11e1-48f5-8cf2-66deae880185 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:08.994874 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2183ef-11e1-48f5-8cf2-66deae880185 admin admin] PATH_INFO: `/v3/projects/ae986c6b484740a9a4629648d1b65e51/users/942c420f56f5449fbae8b4ff780d1834/roles/74a40bc819cb42eea7e966a4b2a016f1` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:09.004650 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 808/1604] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:18:08 2022] PUT /identity/v3/projects/4c435019ac81431cb462873b6eaf636f/users/cdb4258fb5914636a40825b29c6e102e/roles/74a40bc819cb42eea7e966a4b2a016f1 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:18:09.005606 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cf4584-7497-43c8-b6b8-796ebd9334fe admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:09.013125 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2183ef-11e1-48f5-8cf2-66deae880185 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae986c6b484740a9a4629648d1b65e51, user_id=942c420f56f5449fbae8b4ff780d1834, role_id=74a40bc819cb42eea7e966a4b2a016f1)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:09.016320 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2183ef-11e1-48f5-8cf2-66deae880185 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:09.030466 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 797/1605] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:18:08 2022] PUT /identity/v3/projects/ae986c6b484740a9a4629648d1b65e51/users/942c420f56f5449fbae8b4ff780d1834/roles/74a40bc819cb42eea7e966a4b2a016f1 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:18:09.031240 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2183ef-11e1-48f5-8cf2-66deae880185 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:09.067847 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cf4584-7497-43c8-b6b8-796ebd9334fe admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:09.079932 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:18:09.093071 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:18:09.099091 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2183ef-11e1-48f5-8cf2-66deae880185 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:09.104465 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3253b71-4660-43c4-8141-03b1b3f7a1d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:09.105146 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3253b71-4660-43c4-8141-03b1b3f7a1d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:09.105421 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3253b71-4660-43c4-8141-03b1b3f7a1d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:09.105824 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3253b71-4660-43c4-8141-03b1b3f7a1d1 admin admin] PATH_INFO: `/v3/projects/e73db944cba3469fb485c2dccbc56c6c/users/2675b5a0cf2e420c92b8f2055a383131/roles/74a40bc819cb42eea7e966a4b2a016f1` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:09.122911 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3253b71-4660-43c4-8141-03b1b3f7a1d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e73db944cba3469fb485c2dccbc56c6c, user_id=2675b5a0cf2e420c92b8f2055a383131, role_id=74a40bc819cb42eea7e966a4b2a016f1)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:09.125936 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3253b71-4660-43c4-8141-03b1b3f7a1d1 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:09.132755 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02422886-c5cb-4e1b-8180-fb248be37118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:09.133488 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02422886-c5cb-4e1b-8180-fb248be37118 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:09.133843 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02422886-c5cb-4e1b-8180-fb248be37118 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:09.134120 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02422886-c5cb-4e1b-8180-fb248be37118 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:09.134768 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02422886-c5cb-4e1b-8180-fb248be37118 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:09.137143 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02422886-c5cb-4e1b-8180-fb248be37118 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:09.141641 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 798/1606] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:18:09 2022] PUT /identity/v3/projects/e73db944cba3469fb485c2dccbc56c6c/users/2675b5a0cf2e420c92b8f2055a383131/roles/74a40bc819cb42eea7e966a4b2a016f1 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:18:09.142777 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfeb211-515d-42c8-80db-727daf5c4bc3 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:09.143071 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfeb211-515d-42c8-80db-727daf5c4bc3 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:09.143346 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfeb211-515d-42c8-80db-727daf5c4bc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:09.158793 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 809/1607] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:18:09 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:18:09.160288 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9932a1c-9e39-488c-bc24-ed6a6e01a6f3 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:09.160574 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9932a1c-9e39-488c-bc24-ed6a6e01a6f3 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:09.160849 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9932a1c-9e39-488c-bc24-ed6a6e01a6f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:09.171897 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-3dfeb211-515d-42c8-80db-727daf5c4bc3 None None] MFA Rules not processed for user `570f8f1038ca474bba31f467e74a17d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:18:09.173119 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-3dfeb211-515d-42c8-80db-727daf5c4bc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:09.180755 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-a9932a1c-9e39-488c-bc24-ed6a6e01a6f3 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:18:09.181922 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-a9932a1c-9e39-488c-bc24-ed6a6e01a6f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:09.241071 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 810/1608] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:18:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:18:09.242143 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9932a1c-9e39-488c-bc24-ed6a6e01a6f3 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:09.245562 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 799/1609] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:18:09 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:18:09.246371 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfeb211-515d-42c8-80db-727daf5c4bc3 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:09.257808 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfeb211-515d-42c8-80db-727daf5c4bc3 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:09.258260 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9932a1c-9e39-488c-bc24-ed6a6e01a6f3 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:09.262915 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b60812-9f74-4517-a56a-08c69e676c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:09.263326 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baf993a-5512-4f68-b245-8554ecda53cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:09.263567 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b60812-9f74-4517-a56a-08c69e676c58 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:09.263881 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b60812-9f74-4517-a56a-08c69e676c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:09.264020 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baf993a-5512-4f68-b245-8554ecda53cb admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:09.264154 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b60812-9f74-4517-a56a-08c69e676c58 admin admin] PATH_INFO: `/v3/projects/0e3308fc667c4b8c8552e27ba69d40fc/users/8030a8ebdd8c45c7bd30403c866a9278/roles/74a40bc819cb42eea7e966a4b2a016f1` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:09.264296 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baf993a-5512-4f68-b245-8554ecda53cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:09.264567 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baf993a-5512-4f68-b245-8554ecda53cb admin admin] PATH_INFO: `/v3/projects/1e2b975d3d2d45d384b5a8dd92da782f/users/61388384a318473e987ab06426509b69/roles/74a40bc819cb42eea7e966a4b2a016f1` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:09.279592 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b60812-9f74-4517-a56a-08c69e676c58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e3308fc667c4b8c8552e27ba69d40fc, user_id=8030a8ebdd8c45c7bd30403c866a9278, role_id=74a40bc819cb42eea7e966a4b2a016f1)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:09.280003 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baf993a-5512-4f68-b245-8554ecda53cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e2b975d3d2d45d384b5a8dd92da782f, user_id=61388384a318473e987ab06426509b69, role_id=74a40bc819cb42eea7e966a4b2a016f1)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:09.282270 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b60812-9f74-4517-a56a-08c69e676c58 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:09.282630 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baf993a-5512-4f68-b245-8554ecda53cb admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:09.295445 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 800/1610] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:18:09 2022] PUT /identity/v3/projects/0e3308fc667c4b8c8552e27ba69d40fc/users/8030a8ebdd8c45c7bd30403c866a9278/roles/74a40bc819cb42eea7e966a4b2a016f1 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:18:09.296498 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 811/1611] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:18:09 2022] PUT /identity/v3/projects/1e2b975d3d2d45d384b5a8dd92da782f/users/61388384a318473e987ab06426509b69/roles/74a40bc819cb42eea7e966a4b2a016f1 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:18:09.296731 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce796a1c-25e3-4068-a419-0c1a741d04ad None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:09.297025 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce796a1c-25e3-4068-a419-0c1a741d04ad None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:09.297305 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce796a1c-25e3-4068-a419-0c1a741d04ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:09.297608 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b69f422-19ac-47d5-8971-342cae2190df None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:09.297954 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b69f422-19ac-47d5-8971-342cae2190df None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:09.298260 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b69f422-19ac-47d5-8971-342cae2190df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:09.316847 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-6b69f422-19ac-47d5-8971-342cae2190df None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:18:09.318087 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-6b69f422-19ac-47d5-8971-342cae2190df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:09.326170 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-ce796a1c-25e3-4068-a419-0c1a741d04ad None None] MFA Rules not processed for user `cdb4258fb5914636a40825b29c6e102e`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:18:09.327479 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-ce796a1c-25e3-4068-a419-0c1a741d04ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:09.378004 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 812/1612] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:18:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:18:09.379526 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd20185-a1c6-4794-a2eb-3e265ed4fad1 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:09.379841 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd20185-a1c6-4794-a2eb-3e265ed4fad1 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:09.380116 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd20185-a1c6-4794-a2eb-3e265ed4fad1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:09.399066 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 801/1613] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:18:09 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:18:09.406405 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce796a1c-25e3-4068-a419-0c1a741d04ad None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:09.410978 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-efd20185-a1c6-4794-a2eb-3e265ed4fad1 None None] MFA Rules not processed for user `2675b5a0cf2e420c92b8f2055a383131`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:18:09.412234 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-efd20185-a1c6-4794-a2eb-3e265ed4fad1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:09.417585 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce796a1c-25e3-4068-a419-0c1a741d04ad None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:09.422482 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa200ed0-ec46-44b4-a560-fae7812e6735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:09.423366 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa200ed0-ec46-44b4-a560-fae7812e6735 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:09.423659 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa200ed0-ec46-44b4-a560-fae7812e6735 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:09.423935 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa200ed0-ec46-44b4-a560-fae7812e6735 admin admin] PATH_INFO: `/v3/projects/35e74b3e07c6496ba1eeba8a405838f7/users/b00da21ff6d745c8ba6e8ffe43d81755/roles/74a40bc819cb42eea7e966a4b2a016f1` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:09.430275 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:18:09.439142 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa200ed0-ec46-44b4-a560-fae7812e6735 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35e74b3e07c6496ba1eeba8a405838f7, user_id=b00da21ff6d745c8ba6e8ffe43d81755, role_id=74a40bc819cb42eea7e966a4b2a016f1)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:09.442238 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa200ed0-ec46-44b4-a560-fae7812e6735 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:09.455056 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 802/1614] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:18:09 2022] PUT /identity/v3/projects/35e74b3e07c6496ba1eeba8a405838f7/users/b00da21ff6d745c8ba6e8ffe43d81755/roles/74a40bc819cb42eea7e966a4b2a016f1 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:18:09.455961 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa200ed0-ec46-44b4-a560-fae7812e6735 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:09.468404 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:18:09.482792 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 813/1615] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:18:09 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:18:09.483658 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd20185-a1c6-4794-a2eb-3e265ed4fad1 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:09.523867 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa200ed0-ec46-44b4-a560-fae7812e6735 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:09.527150 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae4b390-77db-4a1f-a7d5-d11ee0382104 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:09.527787 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae4b390-77db-4a1f-a7d5-d11ee0382104 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:09.528063 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae4b390-77db-4a1f-a7d5-d11ee0382104 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:09.528334 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae4b390-77db-4a1f-a7d5-d11ee0382104 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:09.539143 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd20185-a1c6-4794-a2eb-3e265ed4fad1 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:09.588705 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b474fa57-068c-435e-9e04-b7749c80fcab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e79498873d4ed29a0e19bbea9b9214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:09.588943 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae4b390-77db-4a1f-a7d5-d11ee0382104 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:09.589339 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b474fa57-068c-435e-9e04-b7749c80fcab service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:09.589598 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b474fa57-068c-435e-9e04-b7749c80fcab service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:09.589866 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b474fa57-068c-435e-9e04-b7749c80fcab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:09.590224 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae4b390-77db-4a1f-a7d5-d11ee0382104 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:09.599843 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 803/1616] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:18:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:18:09.601795 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a3905e-cfa1-496b-b51c-dd022b861644 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:09.602094 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a3905e-cfa1-496b-b51c-dd022b861644 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:09.602370 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a3905e-cfa1-496b-b51c-dd022b861644 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:09.625202 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-79a3905e-cfa1-496b-b51c-dd022b861644 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:18:09.626601 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-79a3905e-cfa1-496b-b51c-dd022b861644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:09.635512 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 804/1617] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:18:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:18:09.636939 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcfb831-f892-4453-a84d-3e3e53eda6d2 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:09.637217 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcfb831-f892-4453-a84d-3e3e53eda6d2 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:09.637483 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcfb831-f892-4453-a84d-3e3e53eda6d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:09.661366 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-cbcfb831-f892-4453-a84d-3e3e53eda6d2 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:18:09.663082 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-cbcfb831-f892-4453-a84d-3e3e53eda6d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:09.664384 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b474fa57-068c-435e-9e04-b7749c80fcab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:09.665860 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b474fa57-068c-435e-9e04-b7749c80fcab service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:09.676284 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 805/1618] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:18:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:18:09.676284 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcfb831-f892-4453-a84d-3e3e53eda6d2 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:09.688785 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 814/1619] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:18:09 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:18:09.689545 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b474fa57-068c-435e-9e04-b7749c80fcab service cinder] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:09.700660 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b474fa57-068c-435e-9e04-b7749c80fcab service cinder] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:09.700848 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcfb831-f892-4453-a84d-3e3e53eda6d2 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:09.705132 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05061538-5db2-4bd7-8162-4c36968622f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e79498873d4ed29a0e19bbea9b9214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:09.706219 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05061538-5db2-4bd7-8162-4c36968622f8 service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:09.706359 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24a1afc-918c-40b8-a913-02e9240ac6e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:09.706805 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05061538-5db2-4bd7-8162-4c36968622f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:09.706805 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05061538-5db2-4bd7-8162-4c36968622f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:09.707272 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24a1afc-918c-40b8-a913-02e9240ac6e2 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:09.707559 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24a1afc-918c-40b8-a913-02e9240ac6e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:09.708317 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24a1afc-918c-40b8-a913-02e9240ac6e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:09.719864 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24a1afc-918c-40b8-a913-02e9240ac6e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:09.721515 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24a1afc-918c-40b8-a913-02e9240ac6e2 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:09.736378 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:18:09.736515 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 806/1620] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:18:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:18:09.739578 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee7bbce-a562-450b-b692-0a33d9d9e97f None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:09.739967 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee7bbce-a562-450b-b692-0a33d9d9e97f None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:09.740310 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee7bbce-a562-450b-b692-0a33d9d9e97f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:09.777753 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-0ee7bbce-a562-450b-b692-0a33d9d9e97f None None] MFA Rules not processed for user `b00da21ff6d745c8ba6e8ffe43d81755`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:18:09.779425 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-0ee7bbce-a562-450b-b692-0a33d9d9e97f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:09.787119 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05061538-5db2-4bd7-8162-4c36968622f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:09.794050 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05061538-5db2-4bd7-8162-4c36968622f8 service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:09.802916 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 815/1621] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:18:09 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:18:09.804526 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05061538-5db2-4bd7-8162-4c36968622f8 service cinder] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:09.820557 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05061538-5db2-4bd7-8162-4c36968622f8 service cinder] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:09.832763 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a447d1e-fb63-4d57-b7d1-595b59a11bb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:09.833678 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a447d1e-fb63-4d57-b7d1-595b59a11bb8 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:09.834175 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a447d1e-fb63-4d57-b7d1-595b59a11bb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:09.834638 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a447d1e-fb63-4d57-b7d1-595b59a11bb8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:09.835802 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a447d1e-fb63-4d57-b7d1-595b59a11bb8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:09.839614 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a447d1e-fb63-4d57-b7d1-595b59a11bb8 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:09.873533 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 807/1622] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:18:09 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:18:09.874566 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee7bbce-a562-450b-b692-0a33d9d9e97f None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:09.879461 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 816/1623] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 19:18:09 2022] POST /identity/v3/projects => generated 405 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:18:09.880313 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a447d1e-fb63-4d57-b7d1-595b59a11bb8 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:09.893132 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee7bbce-a562-450b-b692-0a33d9d9e97f None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:09.907008 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:18:09.953458 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88856fe-6f1c-432a-972e-865bea7ccc2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:09.954459 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88856fe-6f1c-432a-972e-865bea7ccc2c service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:09.955120 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88856fe-6f1c-432a-972e-865bea7ccc2c service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:09.955672 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88856fe-6f1c-432a-972e-865bea7ccc2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:09.962091 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a447d1e-fb63-4d57-b7d1-595b59a11bb8 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:09.969142 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76eacba-ca94-4548-8137-ce1a8cba14b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:09.970028 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76eacba-ca94-4548-8137-ce1a8cba14b2 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:09.970386 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76eacba-ca94-4548-8137-ce1a8cba14b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:09.970697 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76eacba-ca94-4548-8137-ce1a8cba14b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:10.114346 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88856fe-6f1c-432a-972e-865bea7ccc2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:10.117368 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76eacba-ca94-4548-8137-ce1a8cba14b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:10.118831 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88856fe-6f1c-432a-972e-865bea7ccc2c service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:10.121099 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76eacba-ca94-4548-8137-ce1a8cba14b2 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:10.147750 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 808/1624] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:18:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:18:10.148883 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88856fe-6f1c-432a-972e-865bea7ccc2c service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:10.149439 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 817/1625] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:18:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:18:10.150376 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76eacba-ca94-4548-8137-ce1a8cba14b2 service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:10.174062 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76eacba-ca94-4548-8137-ce1a8cba14b2 service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:10.189571 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4f8547-b04b-4a35-a65a-987b272bd14b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:10.190205 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4f8547-b04b-4a35-a65a-987b272bd14b admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:10.191029 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4f8547-b04b-4a35-a65a-987b272bd14b admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:10.200856 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4f8547-b04b-4a35-a65a-987b272bd14b admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:10.203272 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4f8547-b04b-4a35-a65a-987b272bd14b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:10.208987 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4f8547-b04b-4a35-a65a-987b272bd14b admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:10.260100 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 818/1626] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 19:18:10 2022] POST /identity/v3/projects => generated 381 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:18:10.260100 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4f8547-b04b-4a35-a65a-987b272bd14b admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:10.269145 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88856fe-6f1c-432a-972e-865bea7ccc2c service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:10.332037 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:18:10.360452 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68afb5cf-24e8-4b6e-a839-cad3ff12a9ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e79498873d4ed29a0e19bbea9b9214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:10.361079 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68afb5cf-24e8-4b6e-a839-cad3ff12a9ae service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:10.361358 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68afb5cf-24e8-4b6e-a839-cad3ff12a9ae service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:10.361625 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68afb5cf-24e8-4b6e-a839-cad3ff12a9ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:10.418869 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4f8547-b04b-4a35-a65a-987b272bd14b admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:10.425031 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d815a983-ba0a-417a-9f9f-f93c1bbea5e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:10.425764 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d815a983-ba0a-417a-9f9f-f93c1bbea5e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:10.426047 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d815a983-ba0a-417a-9f9f-f93c1bbea5e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:10.426322 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d815a983-ba0a-417a-9f9f-f93c1bbea5e8 admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:10.427038 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d815a983-ba0a-417a-9f9f-f93c1bbea5e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:10.429231 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d815a983-ba0a-417a-9f9f-f93c1bbea5e8 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:10.448053 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:18:10.463329 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 819/1627] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 19:18:10 2022] POST /identity/v3/users => generated 309 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:18:10.465225 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d815a983-ba0a-417a-9f9f-f93c1bbea5e8 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:10.491026 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d815a983-ba0a-417a-9f9f-f93c1bbea5e8 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:10.495972 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68afb5cf-24e8-4b6e-a839-cad3ff12a9ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:10.498051 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68afb5cf-24e8-4b6e-a839-cad3ff12a9ae service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:10.510324 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 809/1628] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:18:10 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:18:10.511424 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68afb5cf-24e8-4b6e-a839-cad3ff12a9ae service cinder] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:10.513569 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1c16c1-2eb4-426c-8cac-91bb1abe2bf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:10.514204 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1c16c1-2eb4-426c-8cac-91bb1abe2bf0 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:10.514483 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1c16c1-2eb4-426c-8cac-91bb1abe2bf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:10.514752 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1c16c1-2eb4-426c-8cac-91bb1abe2bf0 admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:10.515620 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1c16c1-2eb4-426c-8cac-91bb1abe2bf0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:10.517333 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1c16c1-2eb4-426c-8cac-91bb1abe2bf0 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:10.525374 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68afb5cf-24e8-4b6e-a839-cad3ff12a9ae service cinder] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:10.531327 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fcf1ab-c35f-4eb1-b724-ddac4088dab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:10.532047 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fcf1ab-c35f-4eb1-b724-ddac4088dab5 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:10.532659 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fcf1ab-c35f-4eb1-b724-ddac4088dab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:10.533078 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fcf1ab-c35f-4eb1-b724-ddac4088dab5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:10.533830 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fcf1ab-c35f-4eb1-b724-ddac4088dab5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:10.535460 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fcf1ab-c35f-4eb1-b724-ddac4088dab5 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:10.547747 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 810/1629] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:18:10 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:18:10.552350 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fcf1ab-c35f-4eb1-b724-ddac4088dab5 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:10.557676 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 820/1630] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 19:18:10 2022] POST /identity/v3/users => generated 297 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:18:10.562435 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1c16c1-2eb4-426c-8cac-91bb1abe2bf0 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:10.578110 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fcf1ab-c35f-4eb1-b724-ddac4088dab5 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:10.578906 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1c16c1-2eb4-426c-8cac-91bb1abe2bf0 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:10.587856 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1ddf25-c42b-47ab-bb5f-f39cf7ee6720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:10.588713 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1ddf25-c42b-47ab-bb5f-f39cf7ee6720 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:10.588831 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6038321a-d191-42ab-a1c6-e3d59834070e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:10.589063 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1ddf25-c42b-47ab-bb5f-f39cf7ee6720 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:10.589534 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1ddf25-c42b-47ab-bb5f-f39cf7ee6720 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:10.590197 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6038321a-d191-42ab-a1c6-e3d59834070e admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:10.590322 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1ddf25-c42b-47ab-bb5f-f39cf7ee6720 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:10.591953 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6038321a-d191-42ab-a1c6-e3d59834070e admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:10.592342 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6038321a-d191-42ab-a1c6-e3d59834070e admin admin] PATH_INFO: `/v3/projects/26bbca0873214b03a1beb8d8a0035508/users/586cc0af95684f3e9ce30b8e89be309b/roles/163c30c3b3c246beb849ce176f02d9da` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:10.593144 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1ddf25-c42b-47ab-bb5f-f39cf7ee6720 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:10.615062 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 821/1631] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:18:10 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:18:10.616355 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1ddf25-c42b-47ab-bb5f-f39cf7ee6720 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:10.626283 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6038321a-d191-42ab-a1c6-e3d59834070e admin admin] RBAC: Authorizing `identity:create_grant(project_id=26bbca0873214b03a1beb8d8a0035508, user_id=586cc0af95684f3e9ce30b8e89be309b, role_id=163c30c3b3c246beb849ce176f02d9da)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:10.629073 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1ddf25-c42b-47ab-bb5f-f39cf7ee6720 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:10.631841 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6038321a-d191-42ab-a1c6-e3d59834070e admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:10.635820 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2905bf4-9a8c-470e-89aa-e7088d60a260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:10.637562 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2905bf4-9a8c-470e-89aa-e7088d60a260 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:10.637789 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2905bf4-9a8c-470e-89aa-e7088d60a260 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:10.638184 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2905bf4-9a8c-470e-89aa-e7088d60a260 admin admin] PATH_INFO: `/v3/projects/654da2bab7d54ff48312a79db9208f46/users/f4f1590545c44dbbb9dd4e5a8200df93/roles/163c30c3b3c246beb849ce176f02d9da` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:10.660209 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2905bf4-9a8c-470e-89aa-e7088d60a260 admin admin] RBAC: Authorizing `identity:create_grant(project_id=654da2bab7d54ff48312a79db9208f46, user_id=f4f1590545c44dbbb9dd4e5a8200df93, role_id=163c30c3b3c246beb849ce176f02d9da)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:10.663424 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2905bf4-9a8c-470e-89aa-e7088d60a260 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:10.669699 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 811/1632] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:18:10 2022] PUT /identity/v3/projects/26bbca0873214b03a1beb8d8a0035508/users/586cc0af95684f3e9ce30b8e89be309b/roles/163c30c3b3c246beb849ce176f02d9da => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 19:18:10.673406 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6038321a-d191-42ab-a1c6-e3d59834070e admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:10.691620 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 822/1633] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:18:10 2022] PUT /identity/v3/projects/654da2bab7d54ff48312a79db9208f46/users/f4f1590545c44dbbb9dd4e5a8200df93/roles/163c30c3b3c246beb849ce176f02d9da => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:18:10.696292 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2905bf4-9a8c-470e-89aa-e7088d60a260 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:10.815677 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6038321a-d191-42ab-a1c6-e3d59834070e admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:10.845901 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2905bf4-9a8c-470e-89aa-e7088d60a260 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:10.852860 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6baf08b-54cc-4148-9a3c-0a11058c00f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:10.853488 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6baf08b-54cc-4148-9a3c-0a11058c00f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:10.853769 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6baf08b-54cc-4148-9a3c-0a11058c00f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:10.854032 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6baf08b-54cc-4148-9a3c-0a11058c00f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:10.854692 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6baf08b-54cc-4148-9a3c-0a11058c00f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:10.857862 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6baf08b-54cc-4148-9a3c-0a11058c00f4 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:10.877406 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 823/1634] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:18:10 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:18:10.878377 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6baf08b-54cc-4148-9a3c-0a11058c00f4 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:10.896299 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6baf08b-54cc-4148-9a3c-0a11058c00f4 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:10.901812 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe3cba4-cc24-4c0b-a31c-58f6c60bfb07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:10.902496 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe3cba4-cc24-4c0b-a31c-58f6c60bfb07 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:10.902777 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe3cba4-cc24-4c0b-a31c-58f6c60bfb07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:10.903053 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe3cba4-cc24-4c0b-a31c-58f6c60bfb07 admin admin] PATH_INFO: `/v3/projects/654da2bab7d54ff48312a79db9208f46/users/f4f1590545c44dbbb9dd4e5a8200df93/roles/163c30c3b3c246beb849ce176f02d9da` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:10.905809 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe3cba4-cc24-4c0b-a31c-58f6c60bfb07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=654da2bab7d54ff48312a79db9208f46, user_id=f4f1590545c44dbbb9dd4e5a8200df93, role_id=163c30c3b3c246beb849ce176f02d9da)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:10.908880 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe3cba4-cc24-4c0b-a31c-58f6c60bfb07 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:10.932037 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 824/1635] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:18:10 2022] PUT /identity/v3/projects/654da2bab7d54ff48312a79db9208f46/users/f4f1590545c44dbbb9dd4e5a8200df93/roles/163c30c3b3c246beb849ce176f02d9da => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:18:10.934227 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f6a217-5ba2-4f40-bc9c-4c2c84c0ae00 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:10.935176 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f6a217-5ba2-4f40-bc9c-4c2c84c0ae00 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:10.935563 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f6a217-5ba2-4f40-bc9c-4c2c84c0ae00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:10.971510 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b2dc25-00d9-4fe6-b4ca-28cba02d85a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:10.973824 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-c7f6a217-5ba2-4f40-bc9c-4c2c84c0ae00 None None] MFA Rules not processed for user `f4f1590545c44dbbb9dd4e5a8200df93`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:18:10.974561 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b2dc25-00d9-4fe6-b4ca-28cba02d85a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:10.974853 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b2dc25-00d9-4fe6-b4ca-28cba02d85a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:10.975184 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-c7f6a217-5ba2-4f40-bc9c-4c2c84c0ae00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:10.975513 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b2dc25-00d9-4fe6-b4ca-28cba02d85a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:10.978966 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b2dc25-00d9-4fe6-b4ca-28cba02d85a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:10.984542 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b2dc25-00d9-4fe6-b4ca-28cba02d85a2 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:11.009827 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 812/1636] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:18:10 2022] GET /identity/v3/roles => generated 1420 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:18:11.021166 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b2dc25-00d9-4fe6-b4ca-28cba02d85a2 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:11.032211 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b2dc25-00d9-4fe6-b4ca-28cba02d85a2 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:11.036979 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:18:11.106609 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 825/1637] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:18:10 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:18:11.111157 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:18:11.121421 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f6a217-5ba2-4f40-bc9c-4c2c84c0ae00 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:11.128258 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9138a4-fb6e-41d8-b2a7-fe3cee16a0ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:11.129207 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9138a4-fb6e-41d8-b2a7-fe3cee16a0ce admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:11.130277 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9138a4-fb6e-41d8-b2a7-fe3cee16a0ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:11.130795 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9138a4-fb6e-41d8-b2a7-fe3cee16a0ce admin admin] PATH_INFO: `/v3/projects/26bbca0873214b03a1beb8d8a0035508/users/586cc0af95684f3e9ce30b8e89be309b/roles/163c30c3b3c246beb849ce176f02d9da` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:11.133667 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9138a4-fb6e-41d8-b2a7-fe3cee16a0ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=26bbca0873214b03a1beb8d8a0035508, user_id=586cc0af95684f3e9ce30b8e89be309b, role_id=163c30c3b3c246beb849ce176f02d9da)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:11.137635 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9138a4-fb6e-41d8-b2a7-fe3cee16a0ce admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:11.169451 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 813/1638] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:18:11 2022] PUT /identity/v3/projects/26bbca0873214b03a1beb8d8a0035508/users/586cc0af95684f3e9ce30b8e89be309b/roles/163c30c3b3c246beb849ce176f02d9da => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:18:11.181432 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75557ceb-4817-47d4-9df0-2e1ba2181e73 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:11.181723 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75557ceb-4817-47d4-9df0-2e1ba2181e73 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:11.182004 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75557ceb-4817-47d4-9df0-2e1ba2181e73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:11.216642 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-75557ceb-4817-47d4-9df0-2e1ba2181e73 None None] MFA Rules not processed for user `586cc0af95684f3e9ce30b8e89be309b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:18:11.218487 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-75557ceb-4817-47d4-9df0-2e1ba2181e73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:11.230677 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f6a217-5ba2-4f40-bc9c-4c2c84c0ae00 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:11.362187 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feec874d-0fe9-41c9-888b-0a3878e682f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e79498873d4ed29a0e19bbea9b9214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:11.362991 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feec874d-0fe9-41c9-888b-0a3878e682f1 service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:11.363224 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feec874d-0fe9-41c9-888b-0a3878e682f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:11.363498 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feec874d-0fe9-41c9-888b-0a3878e682f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:11.398063 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 814/1639] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:18:11 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:18:11.406888 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75557ceb-4817-47d4-9df0-2e1ba2181e73 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:11.421676 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75557ceb-4817-47d4-9df0-2e1ba2181e73 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:11.440051 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c2aa54-c193-415f-978c-f6620a16d02e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e79498873d4ed29a0e19bbea9b9214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:11.440051 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c2aa54-c193-415f-978c-f6620a16d02e service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:11.440051 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c2aa54-c193-415f-978c-f6620a16d02e service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:11.440051 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c2aa54-c193-415f-978c-f6620a16d02e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:11.459699 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c2aa54-c193-415f-978c-f6620a16d02e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:11.461148 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c2aa54-c193-415f-978c-f6620a16d02e service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:11.474041 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feec874d-0fe9-41c9-888b-0a3878e682f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:11.476929 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feec874d-0fe9-41c9-888b-0a3878e682f1 service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:11.489845 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 826/1640] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:18:11 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 19:18:11.492258 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 815/1641] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:18:11 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 19:18:11.720929 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feec874d-0fe9-41c9-888b-0a3878e682f1 service cinder] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:11.730382 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feec874d-0fe9-41c9-888b-0a3878e682f1 service cinder] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:11.735681 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d50212-40e8-4227-851e-42d65d88c675 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e79498873d4ed29a0e19bbea9b9214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:11.736333 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d50212-40e8-4227-851e-42d65d88c675 service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:11.736613 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d50212-40e8-4227-851e-42d65d88c675 service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:11.736886 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d50212-40e8-4227-851e-42d65d88c675 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:11.762980 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c2aa54-c193-415f-978c-f6620a16d02e service cinder] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:11.790515 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:18:11.814502 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d50212-40e8-4227-851e-42d65d88c675 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:11.817321 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d50212-40e8-4227-851e-42d65d88c675 service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:11.843717 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 827/1642] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:18:11 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:18:11.897722 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c2aa54-c193-415f-978c-f6620a16d02e service cinder] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:11.912537 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fa9daf-8595-406a-b55d-ae4f7c3d84d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:11.913206 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fa9daf-8595-406a-b55d-ae4f7c3d84d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:11.913489 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fa9daf-8595-406a-b55d-ae4f7c3d84d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:11.913933 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fa9daf-8595-406a-b55d-ae4f7c3d84d1 admin admin] PATH_INFO: `/v3/users/2675b5a0cf2e420c92b8f2055a383131` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:11.916057 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fa9daf-8595-406a-b55d-ae4f7c3d84d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2675b5a0cf2e420c92b8f2055a383131)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:11.917525 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: WARNING py.warnings [None req-84fa9daf-8595-406a-b55d-ae4f7c3d84d1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 22 19:18:11.917525 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: warnings.warn(msg) Aug 22 19:18:11.917525 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: Aug 22 19:18:11.919847 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fa9daf-8595-406a-b55d-ae4f7c3d84d1 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:12.115139 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-84fa9daf-8595-406a-b55d-ae4f7c3d84d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2675b5a0cf2e420c92b8f2055a383131'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:12.116521 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-84fa9daf-8595-406a-b55d-ae4f7c3d84d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2675b5a0cf2e420c92b8f2055a383131'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:12.137071 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:18:12.146768 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-84fa9daf-8595-406a-b55d-ae4f7c3d84d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2675b5a0cf2e420c92b8f2055a383131'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:12.166257 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-84fa9daf-8595-406a-b55d-ae4f7c3d84d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2675b5a0cf2e420c92b8f2055a383131'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:12.223423 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-84fa9daf-8595-406a-b55d-ae4f7c3d84d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2675b5a0cf2e420c92b8f2055a383131'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:12.252134 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 816/1643] 10.222.0.61 () {58 vars in 1224 bytes} [Mon Aug 22 19:18:11 2022] DELETE /identity/v3/users/2675b5a0cf2e420c92b8f2055a383131 => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:18:12.256995 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7dd1f1-8dff-4249-a752-66de3539939d None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:12.257747 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7dd1f1-8dff-4249-a752-66de3539939d None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:12.257747 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7dd1f1-8dff-4249-a752-66de3539939d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:12.298238 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-1b7dd1f1-8dff-4249-a752-66de3539939d None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:18:12.299494 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-1b7dd1f1-8dff-4249-a752-66de3539939d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:12.430829 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 828/1644] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:18:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:18:12.446507 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fa9daf-8595-406a-b55d-ae4f7c3d84d1 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:12.448212 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-84fa9daf-8595-406a-b55d-ae4f7c3d84d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:12.529367 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fa9daf-8595-406a-b55d-ae4f7c3d84d1 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:12.537163 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214fc419-2a89-4d5a-9161-9ba42c7a9905 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:12.537806 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214fc419-2a89-4d5a-9161-9ba42c7a9905 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:12.538084 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214fc419-2a89-4d5a-9161-9ba42c7a9905 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:12.538351 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214fc419-2a89-4d5a-9161-9ba42c7a9905 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:12.560873 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214fc419-2a89-4d5a-9161-9ba42c7a9905 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:12.564095 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214fc419-2a89-4d5a-9161-9ba42c7a9905 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:12.593954 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 817/1645] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:18:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:18:13.287069 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61b010e-7be2-4e0d-b8c8-fee991fd0656 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:13.287597 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61b010e-7be2-4e0d-b8c8-fee991fd0656 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:13.287714 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61b010e-7be2-4e0d-b8c8-fee991fd0656 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:13.289506 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 829/1646] 10.222.0.61 () {54 vars in 962 bytes} [Mon Aug 22 19:18:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:18:13.303047 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d75704d-70ad-49f6-bbd9-850157ac5b1a None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:13.303594 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d75704d-70ad-49f6-bbd9-850157ac5b1a None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:13.303594 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d75704d-70ad-49f6-bbd9-850157ac5b1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:13.383779 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-3d75704d-70ad-49f6-bbd9-850157ac5b1a None None] MFA Rules not processed for user `e7f43e2756b746a6825ca1a06b7d7c91`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:18:13.385462 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-3d75704d-70ad-49f6-bbd9-850157ac5b1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:13.505306 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 818/1647] 10.222.0.61 () {58 vars in 1059 bytes} [Mon Aug 22 19:18:13 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:18:13.506604 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2df89c-f4b4-42ed-b2b5-f5206680c80e None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:13.507045 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2df89c-f4b4-42ed-b2b5-f5206680c80e None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:13.507672 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2df89c-f4b4-42ed-b2b5-f5206680c80e None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:13.509027 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 830/1648] 10.222.0.61 () {54 vars in 912 bytes} [Mon Aug 22 19:18:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 19:18:13.517025 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c8a5a3-750a-4371-bcdd-5a76bb7f1127 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:13.518085 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c8a5a3-750a-4371-bcdd-5a76bb7f1127 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:13.518085 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c8a5a3-750a-4371-bcdd-5a76bb7f1127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:13.552334 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-23c8a5a3-750a-4371-bcdd-5a76bb7f1127 None None] MFA Rules not processed for user `e7f43e2756b746a6825ca1a06b7d7c91`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:18:13.553609 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-23c8a5a3-750a-4371-bcdd-5a76bb7f1127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:13.579285 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 819/1649] 10.222.0.61 () {58 vars in 1009 bytes} [Mon Aug 22 19:18:13 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:18:13.591771 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2df89c-f4b4-42ed-b2b5-f5206680c80e None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:13.593631 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-ac2df89c-f4b4-42ed-b2b5-f5206680c80e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:13.733411 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2df89c-f4b4-42ed-b2b5-f5206680c80e None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:13.742305 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462d3dfa-5691-4a53-b8b4-282ac0cfb6fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e79498873d4ed29a0e19bbea9b9214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:13.742952 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462d3dfa-5691-4a53-b8b4-282ac0cfb6fc service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:13.743239 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462d3dfa-5691-4a53-b8b4-282ac0cfb6fc service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:13.743506 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462d3dfa-5691-4a53-b8b4-282ac0cfb6fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:13.770609 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462d3dfa-5691-4a53-b8b4-282ac0cfb6fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:13.772258 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462d3dfa-5691-4a53-b8b4-282ac0cfb6fc service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:13.800155 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 831/1650] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:18:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:18:13.891716 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c8a5a3-750a-4371-bcdd-5a76bb7f1127 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:13.893137 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-23c8a5a3-750a-4371-bcdd-5a76bb7f1127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:13.918048 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c8a5a3-750a-4371-bcdd-5a76bb7f1127 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:13.929717 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2910b334-e10c-4481-88db-b5e72ec20e9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:13.932032 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2910b334-e10c-4481-88db-b5e72ec20e9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:13.934695 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2910b334-e10c-4481-88db-b5e72ec20e9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:13.936183 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2910b334-e10c-4481-88db-b5e72ec20e9d admin admin] PATH_INFO: `/v3/projects/e73db944cba3469fb485c2dccbc56c6c` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:13.940081 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2910b334-e10c-4481-88db-b5e72ec20e9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e73db944cba3469fb485c2dccbc56c6c)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:13.946274 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2910b334-e10c-4481-88db-b5e72ec20e9d admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:14.063585 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:18:14.083599 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-2910b334-e10c-4481-88db-b5e72ec20e9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e73db944cba3469fb485c2dccbc56c6c', 'request_id': 'req-2910b334-e10c-4481-88db-b5e72ec20e9d'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:14.122363 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-2910b334-e10c-4481-88db-b5e72ec20e9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e73db944cba3469fb485c2dccbc56c6c', 'request_id': 'req-2910b334-e10c-4481-88db-b5e72ec20e9d'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:14.133711 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-2910b334-e10c-4481-88db-b5e72ec20e9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e73db944cba3469fb485c2dccbc56c6c', 'request_id': 'req-2910b334-e10c-4481-88db-b5e72ec20e9d'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:14.156827 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-2910b334-e10c-4481-88db-b5e72ec20e9d admin admin] The token cache is being invalidate because project e73db944cba3469fb485c2dccbc56c6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 19:18:14.158914 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-2910b334-e10c-4481-88db-b5e72ec20e9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:14.159776 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-2910b334-e10c-4481-88db-b5e72ec20e9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:14.164702 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 820/1651] 10.222.0.61 () {58 vars in 1233 bytes} [Mon Aug 22 19:18:13 2022] DELETE /identity/v3/projects/e73db944cba3469fb485c2dccbc56c6c => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:18:14.169544 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462d3dfa-5691-4a53-b8b4-282ac0cfb6fc service cinder] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:14.191914 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462d3dfa-5691-4a53-b8b4-282ac0cfb6fc service cinder] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:14.406951 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6127b0-349c-4d08-8411-791b4231e537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:14.408329 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6127b0-349c-4d08-8411-791b4231e537 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:14.409210 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6127b0-349c-4d08-8411-791b4231e537 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:14.410651 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6127b0-349c-4d08-8411-791b4231e537 admin admin] PATH_INFO: `/v3/users/586cc0af95684f3e9ce30b8e89be309b` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:14.414082 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6127b0-349c-4d08-8411-791b4231e537 admin admin] RBAC: Authorizing `identity:delete_user(user_id=586cc0af95684f3e9ce30b8e89be309b)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:14.417843 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6127b0-349c-4d08-8411-791b4231e537 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:14.483987 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:18:14.564521 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-fd6127b0-349c-4d08-8411-791b4231e537 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '586cc0af95684f3e9ce30b8e89be309b'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:14.568123 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-fd6127b0-349c-4d08-8411-791b4231e537 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '586cc0af95684f3e9ce30b8e89be309b'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:14.604586 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-fd6127b0-349c-4d08-8411-791b4231e537 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '586cc0af95684f3e9ce30b8e89be309b'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:14.667677 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-fd6127b0-349c-4d08-8411-791b4231e537 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '586cc0af95684f3e9ce30b8e89be309b'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:14.677483 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-fd6127b0-349c-4d08-8411-791b4231e537 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '586cc0af95684f3e9ce30b8e89be309b'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:14.700928 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 832/1652] 10.222.0.61 () {58 vars in 1224 bytes} [Mon Aug 22 19:18:14 2022] DELETE /identity/v3/users/586cc0af95684f3e9ce30b8e89be309b => generated 0 bytes in 532 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:18:16.256578 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2910b334-e10c-4481-88db-b5e72ec20e9d admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:16.258064 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-2910b334-e10c-4481-88db-b5e72ec20e9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:16.278589 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2910b334-e10c-4481-88db-b5e72ec20e9d admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:16.404405 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6176f2-3170-4864-ae86-5d2cf4eba165 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:16.404902 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6176f2-3170-4864-ae86-5d2cf4eba165 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:16.405138 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6176f2-3170-4864-ae86-5d2cf4eba165 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:16.406783 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 833/1653] 10.222.0.61 () {54 vars in 962 bytes} [Mon Aug 22 19:18:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 19:18:16.415250 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2db65d8-1a37-42bc-a614-f68ff0a62556 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:16.415675 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2db65d8-1a37-42bc-a614-f68ff0a62556 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:16.416182 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2db65d8-1a37-42bc-a614-f68ff0a62556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:16.448631 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fb5bc2-029a-4724-9aba-412359974d7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:16.448631 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fb5bc2-029a-4724-9aba-412359974d7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:16.448631 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fb5bc2-029a-4724-9aba-412359974d7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:16.454991 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fb5bc2-029a-4724-9aba-412359974d7f admin admin] PATH_INFO: `/v3/projects/26bbca0873214b03a1beb8d8a0035508` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:16.455579 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-f2db65d8-1a37-42bc-a614-f68ff0a62556 None None] MFA Rules not processed for user `e7f43e2756b746a6825ca1a06b7d7c91`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:18:16.458624 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fb5bc2-029a-4724-9aba-412359974d7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=26bbca0873214b03a1beb8d8a0035508)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:16.461048 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-f2db65d8-1a37-42bc-a614-f68ff0a62556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:16.482531 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fb5bc2-029a-4724-9aba-412359974d7f admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:16.678007 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-28fb5bc2-029a-4724-9aba-412359974d7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26bbca0873214b03a1beb8d8a0035508', 'request_id': 'req-28fb5bc2-029a-4724-9aba-412359974d7f'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:16.700655 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-28fb5bc2-029a-4724-9aba-412359974d7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26bbca0873214b03a1beb8d8a0035508', 'request_id': 'req-28fb5bc2-029a-4724-9aba-412359974d7f'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:16.712380 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-28fb5bc2-029a-4724-9aba-412359974d7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26bbca0873214b03a1beb8d8a0035508', 'request_id': 'req-28fb5bc2-029a-4724-9aba-412359974d7f'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:16.720907 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 834/1654] 10.222.0.61 () {58 vars in 1059 bytes} [Mon Aug 22 19:18:16 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 19:18:16.725578 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02607c2c-bd28-45e7-b737-17ea70faf41e None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:16.725868 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02607c2c-bd28-45e7-b737-17ea70faf41e None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:16.726135 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02607c2c-bd28-45e7-b737-17ea70faf41e None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:16.729717 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 835/1655] 10.222.0.61 () {54 vars in 912 bytes} [Mon Aug 22 19:18:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:18:16.740569 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f24320-3c15-4e6e-8edd-d4574a956781 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:16.740903 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f24320-3c15-4e6e-8edd-d4574a956781 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:16.741122 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f24320-3c15-4e6e-8edd-d4574a956781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:16.746018 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-28fb5bc2-029a-4724-9aba-412359974d7f admin admin] The token cache is being invalidate because project 26bbca0873214b03a1beb8d8a0035508 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 19:18:16.747212 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-28fb5bc2-029a-4724-9aba-412359974d7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:16.747212 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-28fb5bc2-029a-4724-9aba-412359974d7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:16.748544 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 821/1656] 10.222.0.61 () {58 vars in 1233 bytes} [Mon Aug 22 19:18:16 2022] DELETE /identity/v3/projects/26bbca0873214b03a1beb8d8a0035508 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:18:16.754535 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1473288-e233-4c81-8fb5-5e94601892c1 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:16.754819 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1473288-e233-4c81-8fb5-5e94601892c1 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:16.755106 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1473288-e233-4c81-8fb5-5e94601892c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:16.816989 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-e8f24320-3c15-4e6e-8edd-d4574a956781 None None] MFA Rules not processed for user `e7f43e2756b746a6825ca1a06b7d7c91`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:18:16.829400 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-e8f24320-3c15-4e6e-8edd-d4574a956781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:16.855570 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-b1473288-e233-4c81-8fb5-5e94601892c1 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:18:16.856975 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-b1473288-e233-4c81-8fb5-5e94601892c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:16.991240 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:18:17.014184 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 822/1657] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:18:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:18:17.026440 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008f7076-fa80-490d-8dd9-911d1b7a384a None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:17.027113 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008f7076-fa80-490d-8dd9-911d1b7a384a None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:17.027541 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008f7076-fa80-490d-8dd9-911d1b7a384a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:17.044066 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 836/1658] 10.222.0.61 () {58 vars in 1009 bytes} [Mon Aug 22 19:18:16 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 19:18:17.055631 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f24320-3c15-4e6e-8edd-d4574a956781 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:17.057169 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-e8f24320-3c15-4e6e-8edd-d4574a956781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:17.084626 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-008f7076-fa80-490d-8dd9-911d1b7a384a None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:18:17.086495 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-008f7076-fa80-490d-8dd9-911d1b7a384a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:17.111584 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 823/1659] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:18:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:18:17.112661 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008f7076-fa80-490d-8dd9-911d1b7a384a None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:17.124901 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:18:17.127325 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008f7076-fa80-490d-8dd9-911d1b7a384a None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:17.136152 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41f5cab-fdfc-488e-9ba0-41be6d3be403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:17.137445 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41f5cab-fdfc-488e-9ba0-41be6d3be403 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:17.137828 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41f5cab-fdfc-488e-9ba0-41be6d3be403 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:17.139576 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41f5cab-fdfc-488e-9ba0-41be6d3be403 admin admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:17.140823 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41f5cab-fdfc-488e-9ba0-41be6d3be403 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:18:17.140823 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41f5cab-fdfc-488e-9ba0-41be6d3be403 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:17.148904 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41f5cab-fdfc-488e-9ba0-41be6d3be403 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:17.201296 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 824/1660] 10.222.0.61 () {58 vars in 1153 bytes} [Mon Aug 22 19:18:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:18:17.202404 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41f5cab-fdfc-488e-9ba0-41be6d3be403 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:17.208712 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f24320-3c15-4e6e-8edd-d4574a956781 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:17.219903 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214d9f8c-1ebf-4365-a5a3-b66260c7536e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e79498873d4ed29a0e19bbea9b9214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:17.221379 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214d9f8c-1ebf-4365-a5a3-b66260c7536e service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:17.221850 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214d9f8c-1ebf-4365-a5a3-b66260c7536e service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:17.226123 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214d9f8c-1ebf-4365-a5a3-b66260c7536e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:17.242603 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41f5cab-fdfc-488e-9ba0-41be6d3be403 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:17.256788 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214d9f8c-1ebf-4365-a5a3-b66260c7536e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:17.262783 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214d9f8c-1ebf-4365-a5a3-b66260c7536e service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:17.276774 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35757d53-66d7-4383-b400-9db203b8d5e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:17.277563 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35757d53-66d7-4383-b400-9db203b8d5e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:17.277685 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35757d53-66d7-4383-b400-9db203b8d5e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:17.277962 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35757d53-66d7-4383-b400-9db203b8d5e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:17.280604 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35757d53-66d7-4383-b400-9db203b8d5e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:17.282399 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 837/1661] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:18:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:18:17.282602 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35757d53-66d7-4383-b400-9db203b8d5e8 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:17.428243 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214d9f8c-1ebf-4365-a5a3-b66260c7536e service cinder] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:17.444754 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214d9f8c-1ebf-4365-a5a3-b66260c7536e service cinder] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:17.452926 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 825/1662] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 19:18:17 2022] POST /identity/v3/projects => generated 403 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:18:17.620206 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf5c97d-e85d-43da-9c0c-a6eca0f8a6e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:17.621773 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf5c97d-e85d-43da-9c0c-a6eca0f8a6e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:17.622353 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf5c97d-e85d-43da-9c0c-a6eca0f8a6e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:17.622972 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf5c97d-e85d-43da-9c0c-a6eca0f8a6e9 admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:17.624507 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf5c97d-e85d-43da-9c0c-a6eca0f8a6e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:17.628208 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf5c97d-e85d-43da-9c0c-a6eca0f8a6e9 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:17.699162 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35757d53-66d7-4383-b400-9db203b8d5e8 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:17.711119 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 838/1663] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 19:18:17 2022] POST /identity/v3/users => generated 302 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:18:17.716226 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35757d53-66d7-4383-b400-9db203b8d5e8 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:17.735836 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846b0b38-6992-4c16-a7a5-04c156d6b1a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:17.736879 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846b0b38-6992-4c16-a7a5-04c156d6b1a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:17.738418 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846b0b38-6992-4c16-a7a5-04c156d6b1a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:17.738905 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846b0b38-6992-4c16-a7a5-04c156d6b1a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:17.740097 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846b0b38-6992-4c16-a7a5-04c156d6b1a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:17.749342 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846b0b38-6992-4c16-a7a5-04c156d6b1a7 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:17.773220 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 826/1664] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:18:17 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:18:17.777895 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf5c97d-e85d-43da-9c0c-a6eca0f8a6e9 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:17.803375 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf5c97d-e85d-43da-9c0c-a6eca0f8a6e9 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:17.816389 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae5c069-87c6-434e-88c0-99843686b097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:17.817395 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae5c069-87c6-434e-88c0-99843686b097 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:17.817891 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae5c069-87c6-434e-88c0-99843686b097 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:17.818368 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae5c069-87c6-434e-88c0-99843686b097 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:17.819362 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae5c069-87c6-434e-88c0-99843686b097 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:17.821289 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae5c069-87c6-434e-88c0-99843686b097 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:17.847779 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 839/1665] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:18:17 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:18:17.852074 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846b0b38-6992-4c16-a7a5-04c156d6b1a7 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:17.888363 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846b0b38-6992-4c16-a7a5-04c156d6b1a7 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:17.898960 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ebb998-006a-44d5-9323-1b56dc68a0bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:17.901406 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ebb998-006a-44d5-9323-1b56dc68a0bb admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:17.902927 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ebb998-006a-44d5-9323-1b56dc68a0bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:17.903421 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ebb998-006a-44d5-9323-1b56dc68a0bb admin admin] PATH_INFO: `/v3/projects/11923b7e762449d19d2896eb4a08682d/users/5a7e900f6c91466ab298cf9e6326f227/roles/74a40bc819cb42eea7e966a4b2a016f1` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:17.986975 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ebb998-006a-44d5-9323-1b56dc68a0bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=11923b7e762449d19d2896eb4a08682d, user_id=5a7e900f6c91466ab298cf9e6326f227, role_id=74a40bc819cb42eea7e966a4b2a016f1)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:17.996226 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ebb998-006a-44d5-9323-1b56dc68a0bb admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:18.032182 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 827/1666] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:18:17 2022] PUT /identity/v3/projects/11923b7e762449d19d2896eb4a08682d/users/5a7e900f6c91466ab298cf9e6326f227/roles/74a40bc819cb42eea7e966a4b2a016f1 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:18:18.037267 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca77a2f4-42f5-420c-9e62-d346e341ede9 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:18.039544 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca77a2f4-42f5-420c-9e62-d346e341ede9 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:18.039965 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca77a2f4-42f5-420c-9e62-d346e341ede9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:18.084321 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-ca77a2f4-42f5-420c-9e62-d346e341ede9 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:18:18.085931 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-ca77a2f4-42f5-420c-9e62-d346e341ede9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:18.213707 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 840/1667] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:18:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:18:18.274464 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ebb998-006a-44d5-9323-1b56dc68a0bb admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:18.276337 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-56ebb998-006a-44d5-9323-1b56dc68a0bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:18.427309 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ebb998-006a-44d5-9323-1b56dc68a0bb admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:18.433923 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be277bb9-4ff7-4a28-b6a0-0f3955973594 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:18.434563 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be277bb9-4ff7-4a28-b6a0-0f3955973594 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:18.434840 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be277bb9-4ff7-4a28-b6a0-0f3955973594 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:18.435163 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be277bb9-4ff7-4a28-b6a0-0f3955973594 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:18.450816 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be277bb9-4ff7-4a28-b6a0-0f3955973594 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:18.460490 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be277bb9-4ff7-4a28-b6a0-0f3955973594 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:18.482630 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 828/1668] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:18:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:18:22.908621 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c765a5-4ded-457e-aad3-6719f833223b None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:22.909107 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c765a5-4ded-457e-aad3-6719f833223b None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:22.909498 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c765a5-4ded-457e-aad3-6719f833223b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:22.990475 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-d9c765a5-4ded-457e-aad3-6719f833223b None None] MFA Rules not processed for user `5a7e900f6c91466ab298cf9e6326f227`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:18:22.992786 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-d9c765a5-4ded-457e-aad3-6719f833223b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:23.046621 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:18:23.152729 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 841/1669] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:18:22 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:18:23.154377 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be277bb9-4ff7-4a28-b6a0-0f3955973594 service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:23.168706 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be277bb9-4ff7-4a28-b6a0-0f3955973594 service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:23.175593 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c78068-80e3-464e-a61e-293b6806aefe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:23.176425 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c78068-80e3-464e-a61e-293b6806aefe admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:23.176710 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c78068-80e3-464e-a61e-293b6806aefe admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:23.177076 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c78068-80e3-464e-a61e-293b6806aefe admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:23.178841 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c78068-80e3-464e-a61e-293b6806aefe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:23.181556 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c78068-80e3-464e-a61e-293b6806aefe admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:23.210869 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91457438-d0fd-4060-8618-62b7761f79d4 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:23.214093 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91457438-d0fd-4060-8618-62b7761f79d4 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:23.215271 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91457438-d0fd-4060-8618-62b7761f79d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:23.279422 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 829/1670] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 19:18:23 2022] POST /identity/v3/projects => generated 403 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:18:23.283693 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c78068-80e3-464e-a61e-293b6806aefe admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:23.298889 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-91457438-d0fd-4060-8618-62b7761f79d4 None None] MFA Rules not processed for user `942c420f56f5449fbae8b4ff780d1834`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:18:23.303100 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-91457438-d0fd-4060-8618-62b7761f79d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:23.504520 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c78068-80e3-464e-a61e-293b6806aefe admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:23.523014 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9ec874-b1cc-4b1b-8988-d613960dcd73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:23.524433 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9ec874-b1cc-4b1b-8988-d613960dcd73 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:23.526039 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9ec874-b1cc-4b1b-8988-d613960dcd73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:23.529818 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9ec874-b1cc-4b1b-8988-d613960dcd73 admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:23.532640 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9ec874-b1cc-4b1b-8988-d613960dcd73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:23.536542 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9ec874-b1cc-4b1b-8988-d613960dcd73 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:23.548768 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 842/1671] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:18:23 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:18:23.551975 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960c6a55-3e20-4fb3-b2df-b90ea9a0fd36 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:23.552307 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960c6a55-3e20-4fb3-b2df-b90ea9a0fd36 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:23.552606 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960c6a55-3e20-4fb3-b2df-b90ea9a0fd36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:23.567832 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:18:23.609232 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 830/1672] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 19:18:23 2022] POST /identity/v3/users => generated 308 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:18:23.611914 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9ec874-b1cc-4b1b-8988-d613960dcd73 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:23.616941 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-960c6a55-3e20-4fb3-b2df-b90ea9a0fd36 None None] MFA Rules not processed for user `942c420f56f5449fbae8b4ff780d1834`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:18:23.622125 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-960c6a55-3e20-4fb3-b2df-b90ea9a0fd36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:23.641596 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9ec874-b1cc-4b1b-8988-d613960dcd73 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:23.647988 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 843/1673] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:18:23 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:18:23.657040 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960c6a55-3e20-4fb3-b2df-b90ea9a0fd36 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:23.659985 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-960c6a55-3e20-4fb3-b2df-b90ea9a0fd36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:23.706202 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95c38c9-4b9c-4a54-af69-c8eac20d28bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:23.706880 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95c38c9-4b9c-4a54-af69-c8eac20d28bd admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:23.707159 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95c38c9-4b9c-4a54-af69-c8eac20d28bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:23.707431 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95c38c9-4b9c-4a54-af69-c8eac20d28bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:23.708138 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95c38c9-4b9c-4a54-af69-c8eac20d28bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:23.715579 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95c38c9-4b9c-4a54-af69-c8eac20d28bd admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:23.759459 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 831/1674] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:18:23 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:18:23.771002 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95c38c9-4b9c-4a54-af69-c8eac20d28bd admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:23.822494 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95c38c9-4b9c-4a54-af69-c8eac20d28bd admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:23.858502 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63ff394-8ed8-4de0-a63d-7c8bbdde25d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:23.859201 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63ff394-8ed8-4de0-a63d-7c8bbdde25d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:23.859480 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63ff394-8ed8-4de0-a63d-7c8bbdde25d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:23.859782 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63ff394-8ed8-4de0-a63d-7c8bbdde25d8 admin admin] PATH_INFO: `/v3/projects/cb5bf4323a274f72a3865aa7cc8dce4d/users/6f4f19651fa6459392780b8ab2e1cdf4/roles/163c30c3b3c246beb849ce176f02d9da` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:23.904617 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63ff394-8ed8-4de0-a63d-7c8bbdde25d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb5bf4323a274f72a3865aa7cc8dce4d, user_id=6f4f19651fa6459392780b8ab2e1cdf4, role_id=163c30c3b3c246beb849ce176f02d9da)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:23.914309 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63ff394-8ed8-4de0-a63d-7c8bbdde25d8 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:23.997903 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 832/1675] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:18:23 2022] PUT /identity/v3/projects/cb5bf4323a274f72a3865aa7cc8dce4d/users/6f4f19651fa6459392780b8ab2e1cdf4/roles/163c30c3b3c246beb849ce176f02d9da => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:18:24.022460 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84d1b1c-88a3-4358-8e78-4d01768cb25a None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:24.022925 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84d1b1c-88a3-4358-8e78-4d01768cb25a None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:24.023285 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84d1b1c-88a3-4358-8e78-4d01768cb25a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:24.064603 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960c6a55-3e20-4fb3-b2df-b90ea9a0fd36 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:24.177597 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-c84d1b1c-88a3-4358-8e78-4d01768cb25a None None] MFA Rules not processed for user `61388384a318473e987ab06426509b69`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:18:24.184277 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-c84d1b1c-88a3-4358-8e78-4d01768cb25a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:24.289059 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:18:24.338813 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e1fcb1-f632-472f-82a3-4f3a7a80d505 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f43e2756b746a6825ca1a06b7d7c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:24.338813 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e1fcb1-f632-472f-82a3-4f3a7a80d505 service nova] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:24.338813 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e1fcb1-f632-472f-82a3-4f3a7a80d505 service nova] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:24.339148 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e1fcb1-f632-472f-82a3-4f3a7a80d505 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:24.395236 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 833/1676] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:18:23 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 396 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:18:24.406194 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84d1b1c-88a3-4358-8e78-4d01768cb25a None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:24.482161 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e1fcb1-f632-472f-82a3-4f3a7a80d505 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:24.484680 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e1fcb1-f632-472f-82a3-4f3a7a80d505 service nova] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:24.502616 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84d1b1c-88a3-4358-8e78-4d01768cb25a None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:24.506169 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 844/1677] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:18:23 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 849 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:18:24.508561 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c47fb7-16f2-4dae-ae88-e96187a24a45 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:24.509450 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c47fb7-16f2-4dae-ae88-e96187a24a45 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:24.509737 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c47fb7-16f2-4dae-ae88-e96187a24a45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:24.511151 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d368ddfe-dadf-4677-8c34-018306cafb53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:24.512760 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d368ddfe-dadf-4677-8c34-018306cafb53 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:24.513332 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d368ddfe-dadf-4677-8c34-018306cafb53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:24.513819 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d368ddfe-dadf-4677-8c34-018306cafb53 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:24.527564 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d368ddfe-dadf-4677-8c34-018306cafb53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:24.530120 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d368ddfe-dadf-4677-8c34-018306cafb53 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:24.546172 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 834/1678] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:18:24 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:18:24.547162 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d368ddfe-dadf-4677-8c34-018306cafb53 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:24.550495 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-d368ddfe-dadf-4677-8c34-018306cafb53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:24.570486 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-95c47fb7-16f2-4dae-ae88-e96187a24a45 None None] MFA Rules not processed for user `8030a8ebdd8c45c7bd30403c866a9278`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:18:24.571738 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-95c47fb7-16f2-4dae-ae88-e96187a24a45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:24.596379 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d368ddfe-dadf-4677-8c34-018306cafb53 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:24.603898 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242a4080-f8d1-4579-be40-999177cbff72 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f43e2756b746a6825ca1a06b7d7c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:24.604789 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242a4080-f8d1-4579-be40-999177cbff72 service nova] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:24.605196 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242a4080-f8d1-4579-be40-999177cbff72 service nova] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:24.605574 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242a4080-f8d1-4579-be40-999177cbff72 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:24.616052 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242a4080-f8d1-4579-be40-999177cbff72 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:24.618016 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242a4080-f8d1-4579-be40-999177cbff72 service nova] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:24.635079 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 835/1679] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:18:24 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:18:24.637120 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242a4080-f8d1-4579-be40-999177cbff72 service nova] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:24.650226 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242a4080-f8d1-4579-be40-999177cbff72 service nova] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:24.660126 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4192aa7c-721e-4bb9-8a29-f21215de93ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:24.661241 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4192aa7c-721e-4bb9-8a29-f21215de93ac admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:24.663093 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4192aa7c-721e-4bb9-8a29-f21215de93ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:24.663607 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4192aa7c-721e-4bb9-8a29-f21215de93ac admin admin] PATH_INFO: `/v3/projects/cb5bf4323a274f72a3865aa7cc8dce4d/users/6f4f19651fa6459392780b8ab2e1cdf4/roles/163c30c3b3c246beb849ce176f02d9da` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:24.667038 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4192aa7c-721e-4bb9-8a29-f21215de93ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb5bf4323a274f72a3865aa7cc8dce4d, user_id=6f4f19651fa6459392780b8ab2e1cdf4, role_id=163c30c3b3c246beb849ce176f02d9da)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:24.674765 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4192aa7c-721e-4bb9-8a29-f21215de93ac admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:24.678185 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 845/1680] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:18:24 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:18:24.683628 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74efdcd-3286-4210-9f25-59cfeab598fb None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:24.684112 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74efdcd-3286-4210-9f25-59cfeab598fb None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:24.684386 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74efdcd-3286-4210-9f25-59cfeab598fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:24.690660 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 836/1681] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:18:24 2022] PUT /identity/v3/projects/cb5bf4323a274f72a3865aa7cc8dce4d/users/6f4f19651fa6459392780b8ab2e1cdf4/roles/163c30c3b3c246beb849ce176f02d9da => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:18:24.692772 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074f48b7-acbb-48e3-ab86-4f76054e0a06 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:24.693171 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074f48b7-acbb-48e3-ab86-4f76054e0a06 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:24.693553 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074f48b7-acbb-48e3-ab86-4f76054e0a06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:24.748931 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-e74efdcd-3286-4210-9f25-59cfeab598fb None None] MFA Rules not processed for user `21d0ab6d1a724547b8c643287df897da`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:18:24.750510 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-e74efdcd-3286-4210-9f25-59cfeab598fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:24.792640 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-074f48b7-acbb-48e3-ab86-4f76054e0a06 None None] MFA Rules not processed for user `8030a8ebdd8c45c7bd30403c866a9278`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:18:24.793912 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-074f48b7-acbb-48e3-ab86-4f76054e0a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:24.831025 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:18:24.963741 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 846/1682] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:18:24 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:18:24.968316 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74efdcd-3286-4210-9f25-59cfeab598fb None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:24.988372 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 837/1683] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:18:24 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:18:24.993616 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:18:25.089684 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74efdcd-3286-4210-9f25-59cfeab598fb None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:25.100856 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed3c6d8-293c-4f39-a6f5-7a70c5ad6e1e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f43e2756b746a6825ca1a06b7d7c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:25.102010 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed3c6d8-293c-4f39-a6f5-7a70c5ad6e1e service nova] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:25.102308 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed3c6d8-293c-4f39-a6f5-7a70c5ad6e1e service nova] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:25.102590 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed3c6d8-293c-4f39-a6f5-7a70c5ad6e1e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:25.122055 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed3c6d8-293c-4f39-a6f5-7a70c5ad6e1e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:25.126118 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed3c6d8-293c-4f39-a6f5-7a70c5ad6e1e service nova] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:25.151062 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 847/1684] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:18:24 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:18:25.282377 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074f48b7-acbb-48e3-ab86-4f76054e0a06 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:25.309704 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074f48b7-acbb-48e3-ab86-4f76054e0a06 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:25.321496 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cb114b-c368-47bc-b486-571f3887c863 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f43e2756b746a6825ca1a06b7d7c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:25.322128 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cb114b-c368-47bc-b486-571f3887c863 service nova] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:25.322407 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cb114b-c368-47bc-b486-571f3887c863 service nova] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:25.322677 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cb114b-c368-47bc-b486-571f3887c863 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:25.345681 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cb114b-c368-47bc-b486-571f3887c863 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:25.347309 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cb114b-c368-47bc-b486-571f3887c863 service nova] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:25.383765 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 838/1685] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:18:25 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:18:25.609371 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28b6ad8-e5a6-4374-8446-7771c9d453fc None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:25.609660 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28b6ad8-e5a6-4374-8446-7771c9d453fc None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:25.609933 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28b6ad8-e5a6-4374-8446-7771c9d453fc None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:25.611305 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 848/1686] 10.222.0.61 () {54 vars in 912 bytes} [Mon Aug 22 19:18:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:18:25.616855 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9069b8fd-948f-4494-afc7-9bf08ac742ee None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:25.617141 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9069b8fd-948f-4494-afc7-9bf08ac742ee None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:25.617424 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9069b8fd-948f-4494-afc7-9bf08ac742ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:25.703777 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-9069b8fd-948f-4494-afc7-9bf08ac742ee None None] MFA Rules not processed for user `a4aca06c966a42f29f5c5bc108529d41`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:18:25.705320 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-9069b8fd-948f-4494-afc7-9bf08ac742ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:25.822492 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 839/1687] 10.222.0.61 () {58 vars in 1009 bytes} [Mon Aug 22 19:18:25 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:18:25.876919 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28b6ad8-e5a6-4374-8446-7771c9d453fc None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:25.879859 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-e28b6ad8-e5a6-4374-8446-7771c9d453fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:25.909524 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28b6ad8-e5a6-4374-8446-7771c9d453fc None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:25.919820 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47a1440-630e-47ed-879b-f6004b3708cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:25.920467 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47a1440-630e-47ed-879b-f6004b3708cc service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:25.920745 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47a1440-630e-47ed-879b-f6004b3708cc service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:25.921030 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47a1440-630e-47ed-879b-f6004b3708cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:25.952826 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47a1440-630e-47ed-879b-f6004b3708cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:25.956923 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47a1440-630e-47ed-879b-f6004b3708cc service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:25.996761 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 849/1688] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:18:25 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:18:26.484436 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b61d961-053a-46ac-a4a5-15806649d59a None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:26.484436 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b61d961-053a-46ac-a4a5-15806649d59a None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:26.484436 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b61d961-053a-46ac-a4a5-15806649d59a None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:26.485389 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 840/1689] 10.222.0.61 () {54 vars in 912 bytes} [Mon Aug 22 19:18:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:18:26.498906 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6183a28-1ebc-4cfd-8cd8-dffaa725c3f4 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:26.499395 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6183a28-1ebc-4cfd-8cd8-dffaa725c3f4 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:26.499482 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6183a28-1ebc-4cfd-8cd8-dffaa725c3f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:26.535510 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-a6183a28-1ebc-4cfd-8cd8-dffaa725c3f4 None None] MFA Rules not processed for user `a4aca06c966a42f29f5c5bc108529d41`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:18:26.537934 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-a6183a28-1ebc-4cfd-8cd8-dffaa725c3f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:26.556256 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 850/1690] 10.222.0.61 () {58 vars in 1009 bytes} [Mon Aug 22 19:18:26 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:18:26.592913 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b61d961-053a-46ac-a4a5-15806649d59a None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:26.619041 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b61d961-053a-46ac-a4a5-15806649d59a None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:26.630092 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011312be-381c-4704-99a7-8ce9f4638eda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:26.630900 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011312be-381c-4704-99a7-8ce9f4638eda service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:26.631393 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011312be-381c-4704-99a7-8ce9f4638eda service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:26.631828 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011312be-381c-4704-99a7-8ce9f4638eda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:26.643735 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011312be-381c-4704-99a7-8ce9f4638eda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:26.645618 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011312be-381c-4704-99a7-8ce9f4638eda service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:26.660663 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 841/1691] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:18:26 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:18:27.469220 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a98142a-9991-4142-9083-0609e860ee4c None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:27.470195 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a98142a-9991-4142-9083-0609e860ee4c None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:27.471617 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a98142a-9991-4142-9083-0609e860ee4c None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:27.475680 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 851/1692] 10.222.0.61 () {54 vars in 961 bytes} [Mon Aug 22 19:18:27 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:18:27.485948 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a70c29-0e89-4081-8d52-f9d04e84d67b None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:27.486813 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a70c29-0e89-4081-8d52-f9d04e84d67b None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:27.487374 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a70c29-0e89-4081-8d52-f9d04e84d67b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:27.529704 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-03a70c29-0e89-4081-8d52-f9d04e84d67b None None] MFA Rules not processed for user `eef0f73297e04d62bc22acb790fad6ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:18:27.532531 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-03a70c29-0e89-4081-8d52-f9d04e84d67b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:27.638098 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a78135-999e-4fd7-8ce6-c42670a55b13 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:27.638742 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a78135-999e-4fd7-8ce6-c42670a55b13 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:27.638742 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a78135-999e-4fd7-8ce6-c42670a55b13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:27.773726 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-03a78135-999e-4fd7-8ce6-c42670a55b13 None None] MFA Rules not processed for user `6f4f19651fa6459392780b8ab2e1cdf4`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:18:27.790680 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-03a78135-999e-4fd7-8ce6-c42670a55b13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:27.891554 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 842/1693] 10.222.0.61 () {58 vars in 1058 bytes} [Mon Aug 22 19:18:27 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 407 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 19:18:27.900438 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a70c29-0e89-4081-8d52-f9d04e84d67b None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:27.928865 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a70c29-0e89-4081-8d52-f9d04e84d67b None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:27.938892 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a495fe8c-28b0-4c98-8494-90d5ec04777f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eef0f73297e04d62bc22acb790fad6ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:27.939912 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a495fe8c-28b0-4c98-8494-90d5ec04777f service placement] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:27.940619 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a495fe8c-28b0-4c98-8494-90d5ec04777f service placement] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:27.941053 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a495fe8c-28b0-4c98-8494-90d5ec04777f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:27.948227 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-a495fe8c-28b0-4c98-8494-90d5ec04777f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:28.007930 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a495fe8c-28b0-4c98-8494-90d5ec04777f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:28.018019 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a495fe8c-28b0-4c98-8494-90d5ec04777f service placement] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:28.040228 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 843/1694] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:18:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:18:28.075343 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 852/1695] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:18:27 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 440 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:18:28.847047 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187731a8-fed4-47b3-8cf9-269505a5f557 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:28.847605 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187731a8-fed4-47b3-8cf9-269505a5f557 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:28.847605 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187731a8-fed4-47b3-8cf9-269505a5f557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:28.905692 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-187731a8-fed4-47b3-8cf9-269505a5f557 None None] MFA Rules not processed for user `8030a8ebdd8c45c7bd30403c866a9278`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:18:28.907384 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-187731a8-fed4-47b3-8cf9-269505a5f557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:28.940746 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 844/1696] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:18:28 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:18:29.800201 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a527ef-0c14-4824-a400-b65eef1ef060 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:29.800885 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a527ef-0c14-4824-a400-b65eef1ef060 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:29.800885 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a527ef-0c14-4824-a400-b65eef1ef060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:29.858390 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-40a527ef-0c14-4824-a400-b65eef1ef060 None None] MFA Rules not processed for user `21d0ab6d1a724547b8c643287df897da`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:18:29.860108 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-40a527ef-0c14-4824-a400-b65eef1ef060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:29.886572 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 853/1697] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:18:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:18:31.171635 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187731a8-fed4-47b3-8cf9-269505a5f557 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:31.174924 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-187731a8-fed4-47b3-8cf9-269505a5f557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:31.193733 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:18:31.277386 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4ff8dc-e725-4ad4-8c7f-4407a5a768f1 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:31.277891 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4ff8dc-e725-4ad4-8c7f-4407a5a768f1 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:31.278082 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4ff8dc-e725-4ad4-8c7f-4407a5a768f1 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:31.279861 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 854/1698] 10.222.0.61 () {54 vars in 921 bytes} [Mon Aug 22 19:18:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:18:31.322856 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfab037b-4a45-4a14-9bc1-6c47b64cb019 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:31.324211 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfab037b-4a45-4a14-9bc1-6c47b64cb019 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:31.325050 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfab037b-4a45-4a14-9bc1-6c47b64cb019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:31.332160 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187731a8-fed4-47b3-8cf9-269505a5f557 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:31.339468 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2ade75-f692-4746-915e-389c335b33eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:31.343662 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2ade75-f692-4746-915e-389c335b33eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:31.344194 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2ade75-f692-4746-915e-389c335b33eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:31.344617 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2ade75-f692-4746-915e-389c335b33eb admin admin] PATH_INFO: `/v3/users/b00da21ff6d745c8ba6e8ffe43d81755` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:31.349556 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2ade75-f692-4746-915e-389c335b33eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=b00da21ff6d745c8ba6e8ffe43d81755)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:31.354554 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2ade75-f692-4746-915e-389c335b33eb admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:31.361374 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-bfab037b-4a45-4a14-9bc1-6c47b64cb019 None None] MFA Rules not processed for user `eef0f73297e04d62bc22acb790fad6ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:18:31.364164 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-bfab037b-4a45-4a14-9bc1-6c47b64cb019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:31.393971 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 855/1699] 10.222.0.61 () {58 vars in 1018 bytes} [Mon Aug 22 19:18:31 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 19:18:31.450386 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfab037b-4a45-4a14-9bc1-6c47b64cb019 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:31.453080 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-bfab037b-4a45-4a14-9bc1-6c47b64cb019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:31.505679 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfab037b-4a45-4a14-9bc1-6c47b64cb019 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:31.523304 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-9e2ade75-f692-4746-915e-389c335b33eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b00da21ff6d745c8ba6e8ffe43d81755'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:31.524223 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-9e2ade75-f692-4746-915e-389c335b33eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b00da21ff6d745c8ba6e8ffe43d81755'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:31.525234 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e37bf74-0bc9-4fcc-ad75-1c904b4801bd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eef0f73297e04d62bc22acb790fad6ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:31.530060 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e37bf74-0bc9-4fcc-ad75-1c904b4801bd service placement] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:31.534454 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e37bf74-0bc9-4fcc-ad75-1c904b4801bd service placement] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:31.537757 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e37bf74-0bc9-4fcc-ad75-1c904b4801bd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:31.553176 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-8e37bf74-0bc9-4fcc-ad75-1c904b4801bd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:31.572870 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-9e2ade75-f692-4746-915e-389c335b33eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b00da21ff6d745c8ba6e8ffe43d81755'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:31.595608 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e37bf74-0bc9-4fcc-ad75-1c904b4801bd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:31.601079 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-9e2ade75-f692-4746-915e-389c335b33eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b00da21ff6d745c8ba6e8ffe43d81755'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:31.606581 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e37bf74-0bc9-4fcc-ad75-1c904b4801bd service placement] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:31.632072 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-9e2ade75-f692-4746-915e-389c335b33eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b00da21ff6d745c8ba6e8ffe43d81755'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:31.640434 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 856/1700] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:18:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 19:18:31.643578 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 845/1701] 10.222.0.61 () {58 vars in 1224 bytes} [Mon Aug 22 19:18:31 2022] DELETE /identity/v3/users/b00da21ff6d745c8ba6e8ffe43d81755 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:18:31.647915 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839365bb-119f-487e-a3b7-3c2ffc544d62 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:31.648715 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839365bb-119f-487e-a3b7-3c2ffc544d62 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:31.649368 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839365bb-119f-487e-a3b7-3c2ffc544d62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:31.694631 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-839365bb-119f-487e-a3b7-3c2ffc544d62 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:18:31.698708 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-839365bb-119f-487e-a3b7-3c2ffc544d62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:31.948577 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:18:32.061164 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 857/1702] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:18:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:18:32.243503 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b18e83-1476-46b2-bc43-c56293f86923 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:32.245692 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b18e83-1476-46b2-bc43-c56293f86923 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:32.246171 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b18e83-1476-46b2-bc43-c56293f86923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:32.272250 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839365bb-119f-487e-a3b7-3c2ffc544d62 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:32.274227 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-839365bb-119f-487e-a3b7-3c2ffc544d62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:32.331128 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-b9b18e83-1476-46b2-bc43-c56293f86923 None None] MFA Rules not processed for user `61388384a318473e987ab06426509b69`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:18:32.337083 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-b9b18e83-1476-46b2-bc43-c56293f86923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:32.472870 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839365bb-119f-487e-a3b7-3c2ffc544d62 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:32.480420 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22067d49-4fa0-4c53-b291-478a0fd06440 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:32.481494 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22067d49-4fa0-4c53-b291-478a0fd06440 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:32.481920 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22067d49-4fa0-4c53-b291-478a0fd06440 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:32.482350 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22067d49-4fa0-4c53-b291-478a0fd06440 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:32.509043 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22067d49-4fa0-4c53-b291-478a0fd06440 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:32.511827 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22067d49-4fa0-4c53-b291-478a0fd06440 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:32.533822 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 846/1703] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:18:32 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:18:32.537643 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aebd20f-5d29-479c-a2b3-6a2d35a0a523 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:32.537939 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aebd20f-5d29-479c-a2b3-6a2d35a0a523 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:32.540771 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aebd20f-5d29-479c-a2b3-6a2d35a0a523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:32.542956 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 858/1704] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:18:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:18:32.665542 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-4aebd20f-5d29-479c-a2b3-6a2d35a0a523 None None] MFA Rules not processed for user `399396a8fe254da199bc48b0ca5243d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:18:32.666852 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-4aebd20f-5d29-479c-a2b3-6a2d35a0a523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:32.671820 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:18:32.779869 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 847/1705] 10.222.0.61 () {58 vars in 1009 bytes} [Mon Aug 22 19:18:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:18:33.051480 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22067d49-4fa0-4c53-b291-478a0fd06440 service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:33.053309 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-22067d49-4fa0-4c53-b291-478a0fd06440 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:33.256097 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22067d49-4fa0-4c53-b291-478a0fd06440 service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:33.264058 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fab0732-0b9a-4364-a268-034e4383c1cc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2824359491934d0dacaa21a718becee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:33.270354 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fab0732-0b9a-4364-a268-034e4383c1cc service swift] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:33.279909 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fab0732-0b9a-4364-a268-034e4383c1cc service swift] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:33.280582 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fab0732-0b9a-4364-a268-034e4383c1cc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:33.305991 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fab0732-0b9a-4364-a268-034e4383c1cc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:33.313562 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fab0732-0b9a-4364-a268-034e4383c1cc service swift] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:33.340977 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 859/1706] 10.222.0.61 () {60 vars in 1384 bytes} [Mon Aug 22 19:18:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 22 19:18:35.997814 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dccd258-368b-437c-9a9f-b426197d56d2 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:35.999178 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dccd258-368b-437c-9a9f-b426197d56d2 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:35.999723 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dccd258-368b-437c-9a9f-b426197d56d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:36.080939 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-5dccd258-368b-437c-9a9f-b426197d56d2 None None] MFA Rules not processed for user `a4aca06c966a42f29f5c5bc108529d41`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:18:36.085661 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-5dccd258-368b-437c-9a9f-b426197d56d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:36.105536 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 848/1707] 10.222.0.16 () {58 vars in 1015 bytes} [Mon Aug 22 19:18:35 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:18:36.184626 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fab0732-0b9a-4364-a268-034e4383c1cc service swift] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:36.186119 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-9fab0732-0b9a-4364-a268-034e4383c1cc service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:36.231350 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dccd258-368b-437c-9a9f-b426197d56d2 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:36.233973 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-5dccd258-368b-437c-9a9f-b426197d56d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:36.242775 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fab0732-0b9a-4364-a268-034e4383c1cc service swift] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:36.249156 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2fca6a-17e2-45c9-a0f4-b0a4e623644b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:36.250310 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2fca6a-17e2-45c9-a0f4-b0a4e623644b service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:36.250851 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2fca6a-17e2-45c9-a0f4-b0a4e623644b service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:36.251327 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2fca6a-17e2-45c9-a0f4-b0a4e623644b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:36.269443 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2fca6a-17e2-45c9-a0f4-b0a4e623644b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:36.273545 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2fca6a-17e2-45c9-a0f4-b0a4e623644b service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:36.290608 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 860/1708] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:18:36 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:18:36.294518 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dccd258-368b-437c-9a9f-b426197d56d2 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:36.318427 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159b4885-f7ef-48c8-82bc-fe79a4252937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:36.319619 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159b4885-f7ef-48c8-82bc-fe79a4252937 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:36.320545 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159b4885-f7ef-48c8-82bc-fe79a4252937 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:36.322398 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159b4885-f7ef-48c8-82bc-fe79a4252937 admin admin] PATH_INFO: `/v3/users/5a7e900f6c91466ab298cf9e6326f227` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:36.325897 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159b4885-f7ef-48c8-82bc-fe79a4252937 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a7e900f6c91466ab298cf9e6326f227)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:36.346716 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159b4885-f7ef-48c8-82bc-fe79a4252937 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:36.400937 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2fca6a-17e2-45c9-a0f4-b0a4e623644b service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:36.402542 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-8f2fca6a-17e2-45c9-a0f4-b0a4e623644b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:36.468310 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2fca6a-17e2-45c9-a0f4-b0a4e623644b service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:36.497386 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e91f903-95d0-44d1-93e2-28973b41a52a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:36.498525 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e91f903-95d0-44d1-93e2-28973b41a52a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:36.499964 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e91f903-95d0-44d1-93e2-28973b41a52a admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:36.500293 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e91f903-95d0-44d1-93e2-28973b41a52a admin admin] PATH_INFO: `/v3/projects/35e74b3e07c6496ba1eeba8a405838f7` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:36.504616 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e91f903-95d0-44d1-93e2-28973b41a52a admin admin] RBAC: Authorizing `identity:delete_project(project_id=35e74b3e07c6496ba1eeba8a405838f7)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:36.511593 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: WARNING py.warnings [None req-1e91f903-95d0-44d1-93e2-28973b41a52a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 22 19:18:36.511593 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: warnings.warn(msg) Aug 22 19:18:36.511593 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: Aug 22 19:18:36.527976 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e91f903-95d0-44d1-93e2-28973b41a52a admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:36.596794 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-159b4885-f7ef-48c8-82bc-fe79a4252937 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a7e900f6c91466ab298cf9e6326f227'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:36.598226 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-159b4885-f7ef-48c8-82bc-fe79a4252937 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a7e900f6c91466ab298cf9e6326f227'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:36.657347 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-159b4885-f7ef-48c8-82bc-fe79a4252937 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a7e900f6c91466ab298cf9e6326f227'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:36.674138 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-159b4885-f7ef-48c8-82bc-fe79a4252937 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a7e900f6c91466ab298cf9e6326f227'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:36.711565 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-159b4885-f7ef-48c8-82bc-fe79a4252937 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a7e900f6c91466ab298cf9e6326f227'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:36.725676 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 849/1709] 10.222.0.61 () {58 vars in 1224 bytes} [Mon Aug 22 19:18:36 2022] DELETE /identity/v3/users/5a7e900f6c91466ab298cf9e6326f227 => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:18:36.728862 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:18:36.822696 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-1e91f903-95d0-44d1-93e2-28973b41a52a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35e74b3e07c6496ba1eeba8a405838f7', 'request_id': 'req-1e91f903-95d0-44d1-93e2-28973b41a52a'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:36.846436 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-1e91f903-95d0-44d1-93e2-28973b41a52a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35e74b3e07c6496ba1eeba8a405838f7', 'request_id': 'req-1e91f903-95d0-44d1-93e2-28973b41a52a'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:36.901526 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-1e91f903-95d0-44d1-93e2-28973b41a52a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35e74b3e07c6496ba1eeba8a405838f7', 'request_id': 'req-1e91f903-95d0-44d1-93e2-28973b41a52a'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:36.933295 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-1e91f903-95d0-44d1-93e2-28973b41a52a admin admin] The token cache is being invalidate because project 35e74b3e07c6496ba1eeba8a405838f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 19:18:36.933490 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-1e91f903-95d0-44d1-93e2-28973b41a52a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:36.935503 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-1e91f903-95d0-44d1-93e2-28973b41a52a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:36.938031 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 861/1710] 10.222.0.61 () {58 vars in 1233 bytes} [Mon Aug 22 19:18:36 2022] DELETE /identity/v3/projects/35e74b3e07c6496ba1eeba8a405838f7 => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:18:36.943613 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159b4885-f7ef-48c8-82bc-fe79a4252937 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:36.945226 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-159b4885-f7ef-48c8-82bc-fe79a4252937 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:37.140068 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159b4885-f7ef-48c8-82bc-fe79a4252937 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:37.157043 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e91f903-95d0-44d1-93e2-28973b41a52a admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:37.158728 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-1e91f903-95d0-44d1-93e2-28973b41a52a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:37.181528 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b58cfe-6de3-4b1c-96f8-0d6e95f6e8d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:37.182387 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b58cfe-6de3-4b1c-96f8-0d6e95f6e8d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:37.182680 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b58cfe-6de3-4b1c-96f8-0d6e95f6e8d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:37.182979 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b58cfe-6de3-4b1c-96f8-0d6e95f6e8d6 admin admin] PATH_INFO: `/v3/users/f4f1590545c44dbbb9dd4e5a8200df93` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:37.185726 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b58cfe-6de3-4b1c-96f8-0d6e95f6e8d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4f1590545c44dbbb9dd4e5a8200df93)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:37.195085 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b58cfe-6de3-4b1c-96f8-0d6e95f6e8d6 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:37.247319 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:18:37.332154 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-73b58cfe-6de3-4b1c-96f8-0d6e95f6e8d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4f1590545c44dbbb9dd4e5a8200df93'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:37.334262 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-73b58cfe-6de3-4b1c-96f8-0d6e95f6e8d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4f1590545c44dbbb9dd4e5a8200df93'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:37.352681 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-73b58cfe-6de3-4b1c-96f8-0d6e95f6e8d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4f1590545c44dbbb9dd4e5a8200df93'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:37.369982 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-73b58cfe-6de3-4b1c-96f8-0d6e95f6e8d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4f1590545c44dbbb9dd4e5a8200df93'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:37.378197 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e91f903-95d0-44d1-93e2-28973b41a52a admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:37.390084 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5de057-5ccf-4f52-9a94-3444e92dc226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:37.391012 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5de057-5ccf-4f52-9a94-3444e92dc226 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:37.392380 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5de057-5ccf-4f52-9a94-3444e92dc226 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:37.393808 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5de057-5ccf-4f52-9a94-3444e92dc226 admin admin] PATH_INFO: `/v3/users/cdb4258fb5914636a40825b29c6e102e` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:37.395977 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5de057-5ccf-4f52-9a94-3444e92dc226 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cdb4258fb5914636a40825b29c6e102e)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:37.399180 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5de057-5ccf-4f52-9a94-3444e92dc226 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:37.400779 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-73b58cfe-6de3-4b1c-96f8-0d6e95f6e8d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4f1590545c44dbbb9dd4e5a8200df93'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:37.414238 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 850/1711] 10.222.0.61 () {58 vars in 1224 bytes} [Mon Aug 22 19:18:36 2022] DELETE /identity/v3/users/f4f1590545c44dbbb9dd4e5a8200df93 => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:18:37.518339 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-ae5de057-5ccf-4f52-9a94-3444e92dc226 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cdb4258fb5914636a40825b29c6e102e'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:37.520275 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-ae5de057-5ccf-4f52-9a94-3444e92dc226 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cdb4258fb5914636a40825b29c6e102e'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:37.532831 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-ae5de057-5ccf-4f52-9a94-3444e92dc226 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cdb4258fb5914636a40825b29c6e102e'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:37.544307 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-ae5de057-5ccf-4f52-9a94-3444e92dc226 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cdb4258fb5914636a40825b29c6e102e'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:37.571845 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-ae5de057-5ccf-4f52-9a94-3444e92dc226 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cdb4258fb5914636a40825b29c6e102e'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:37.597016 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 862/1712] 10.222.0.61 () {58 vars in 1224 bytes} [Mon Aug 22 19:18:37 2022] DELETE /identity/v3/users/cdb4258fb5914636a40825b29c6e102e => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:18:37.602505 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c086e9ce-17bc-40af-bfa4-54af2d4e9029 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:37.602969 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c086e9ce-17bc-40af-bfa4-54af2d4e9029 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:37.603252 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c086e9ce-17bc-40af-bfa4-54af2d4e9029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:37.644224 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-c086e9ce-17bc-40af-bfa4-54af2d4e9029 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:18:37.647688 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-c086e9ce-17bc-40af-bfa4-54af2d4e9029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:37.842920 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 851/1713] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:18:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 19:18:37.900659 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5de057-5ccf-4f52-9a94-3444e92dc226 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:37.921273 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-ae5de057-5ccf-4f52-9a94-3444e92dc226 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:38.103399 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:18:38.203595 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5de057-5ccf-4f52-9a94-3444e92dc226 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:38.219261 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621d886d-7071-4e0c-a765-3b64b9143e86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:38.220172 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c086e9ce-17bc-40af-bfa4-54af2d4e9029 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:38.222685 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-c086e9ce-17bc-40af-bfa4-54af2d4e9029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:38.237773 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621d886d-7071-4e0c-a765-3b64b9143e86 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:38.242410 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621d886d-7071-4e0c-a765-3b64b9143e86 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:38.243128 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621d886d-7071-4e0c-a765-3b64b9143e86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:38.252596 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c086e9ce-17bc-40af-bfa4-54af2d4e9029 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:38.269588 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c976237-0f78-40ce-b3fa-c14626466917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:38.270565 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c976237-0f78-40ce-b3fa-c14626466917 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:38.270854 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c976237-0f78-40ce-b3fa-c14626466917 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:38.271163 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c976237-0f78-40ce-b3fa-c14626466917 admin admin] PATH_INFO: `/v3/projects/11923b7e762449d19d2896eb4a08682d` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:38.273370 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c976237-0f78-40ce-b3fa-c14626466917 admin admin] RBAC: Authorizing `identity:delete_project(project_id=11923b7e762449d19d2896eb4a08682d)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:38.279116 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c976237-0f78-40ce-b3fa-c14626466917 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:38.279887 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621d886d-7071-4e0c-a765-3b64b9143e86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:38.281571 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621d886d-7071-4e0c-a765-3b64b9143e86 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:38.322519 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 863/1714] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:18:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:18:38.425324 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-7c976237-0f78-40ce-b3fa-c14626466917 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11923b7e762449d19d2896eb4a08682d', 'request_id': 'req-7c976237-0f78-40ce-b3fa-c14626466917'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:38.471763 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-7c976237-0f78-40ce-b3fa-c14626466917 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11923b7e762449d19d2896eb4a08682d', 'request_id': 'req-7c976237-0f78-40ce-b3fa-c14626466917'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:38.492099 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-7c976237-0f78-40ce-b3fa-c14626466917 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11923b7e762449d19d2896eb4a08682d', 'request_id': 'req-7c976237-0f78-40ce-b3fa-c14626466917'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:38.535702 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-7c976237-0f78-40ce-b3fa-c14626466917 admin admin] The token cache is being invalidate because project 11923b7e762449d19d2896eb4a08682d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 19:18:38.536004 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-7c976237-0f78-40ce-b3fa-c14626466917 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:38.538612 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-7c976237-0f78-40ce-b3fa-c14626466917 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:38.546128 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 852/1715] 10.222.0.61 () {58 vars in 1233 bytes} [Mon Aug 22 19:18:38 2022] DELETE /identity/v3/projects/11923b7e762449d19d2896eb4a08682d => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:18:38.547225 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621d886d-7071-4e0c-a765-3b64b9143e86 service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:38.573025 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621d886d-7071-4e0c-a765-3b64b9143e86 service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:38.685646 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c42ed9-6640-474b-9f07-126f40a4522f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:38.686565 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c42ed9-6640-474b-9f07-126f40a4522f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:38.686890 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c42ed9-6640-474b-9f07-126f40a4522f admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:38.687168 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c42ed9-6640-474b-9f07-126f40a4522f admin admin] PATH_INFO: `/v3/users/6f4f19651fa6459392780b8ab2e1cdf4` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:38.689706 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c42ed9-6640-474b-9f07-126f40a4522f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f4f19651fa6459392780b8ab2e1cdf4)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:38.692839 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c42ed9-6640-474b-9f07-126f40a4522f admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:38.924673 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-16c42ed9-6640-474b-9f07-126f40a4522f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f4f19651fa6459392780b8ab2e1cdf4'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:38.926021 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-16c42ed9-6640-474b-9f07-126f40a4522f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f4f19651fa6459392780b8ab2e1cdf4'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:38.979637 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-16c42ed9-6640-474b-9f07-126f40a4522f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f4f19651fa6459392780b8ab2e1cdf4'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:39.033465 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-16c42ed9-6640-474b-9f07-126f40a4522f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f4f19651fa6459392780b8ab2e1cdf4'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:39.128923 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-16c42ed9-6640-474b-9f07-126f40a4522f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f4f19651fa6459392780b8ab2e1cdf4'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:39.180733 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 864/1716] 10.222.0.61 () {58 vars in 1224 bytes} [Mon Aug 22 19:18:38 2022] DELETE /identity/v3/users/6f4f19651fa6459392780b8ab2e1cdf4 => generated 0 bytes in 631 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 19:18:40.357912 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c976237-0f78-40ce-b3fa-c14626466917 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:40.359556 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-7c976237-0f78-40ce-b3fa-c14626466917 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:40.393646 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c976237-0f78-40ce-b3fa-c14626466917 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:40.426648 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:18:40.624707 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b93ee3-4827-4fc3-8f50-0406823f1f0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:40.625302 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b93ee3-4827-4fc3-8f50-0406823f1f0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:40.625583 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b93ee3-4827-4fc3-8f50-0406823f1f0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:40.625866 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b93ee3-4827-4fc3-8f50-0406823f1f0d admin admin] PATH_INFO: `/v3/projects/654da2bab7d54ff48312a79db9208f46` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:40.632896 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b93ee3-4827-4fc3-8f50-0406823f1f0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=654da2bab7d54ff48312a79db9208f46)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:40.634907 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b93ee3-4827-4fc3-8f50-0406823f1f0d admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:40.759522 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c42ed9-6640-474b-9f07-126f40a4522f admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:40.763265 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-16c42ed9-6640-474b-9f07-126f40a4522f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:40.809176 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c42ed9-6640-474b-9f07-126f40a4522f admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:40.846478 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-c2b93ee3-4827-4fc3-8f50-0406823f1f0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '654da2bab7d54ff48312a79db9208f46', 'request_id': 'req-c2b93ee3-4827-4fc3-8f50-0406823f1f0d'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:40.867624 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-c2b93ee3-4827-4fc3-8f50-0406823f1f0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '654da2bab7d54ff48312a79db9208f46', 'request_id': 'req-c2b93ee3-4827-4fc3-8f50-0406823f1f0d'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:40.883551 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-c2b93ee3-4827-4fc3-8f50-0406823f1f0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '654da2bab7d54ff48312a79db9208f46', 'request_id': 'req-c2b93ee3-4827-4fc3-8f50-0406823f1f0d'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:40.914149 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-c2b93ee3-4827-4fc3-8f50-0406823f1f0d admin admin] The token cache is being invalidate because project 654da2bab7d54ff48312a79db9208f46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 19:18:40.914384 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-c2b93ee3-4827-4fc3-8f50-0406823f1f0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:40.915133 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-c2b93ee3-4827-4fc3-8f50-0406823f1f0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:40.920965 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 853/1717] 10.222.0.61 () {58 vars in 1233 bytes} [Mon Aug 22 19:18:40 2022] DELETE /identity/v3/projects/654da2bab7d54ff48312a79db9208f46 => generated 0 bytes in 560 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:18:40.922475 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbadd0ec-4b14-4cb4-8f4d-ecabac914c7f None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:40.922770 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbadd0ec-4b14-4cb4-8f4d-ecabac914c7f None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:40.923052 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbadd0ec-4b14-4cb4-8f4d-ecabac914c7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:40.977477 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-bbadd0ec-4b14-4cb4-8f4d-ecabac914c7f None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:18:40.978660 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-bbadd0ec-4b14-4cb4-8f4d-ecabac914c7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:41.028044 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4673c7ca-2c93-4def-998b-85b716e00ed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:41.028798 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4673c7ca-2c93-4def-998b-85b716e00ed5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:41.028892 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4673c7ca-2c93-4def-998b-85b716e00ed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:41.029260 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4673c7ca-2c93-4def-998b-85b716e00ed5 admin admin] PATH_INFO: `/v3/projects/cb5bf4323a274f72a3865aa7cc8dce4d` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:41.030414 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4673c7ca-2c93-4def-998b-85b716e00ed5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb5bf4323a274f72a3865aa7cc8dce4d)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:41.035486 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4673c7ca-2c93-4def-998b-85b716e00ed5 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:41.040564 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:18:41.127081 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 854/1718] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:18:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:18:41.129975 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0ecf08-f261-4526-9c5f-c4a19b7698b5 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:41.132754 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0ecf08-f261-4526-9c5f-c4a19b7698b5 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:41.133531 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0ecf08-f261-4526-9c5f-c4a19b7698b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:41.155618 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-4673c7ca-2c93-4def-998b-85b716e00ed5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb5bf4323a274f72a3865aa7cc8dce4d', 'request_id': 'req-4673c7ca-2c93-4def-998b-85b716e00ed5'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:41.170943 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-4673c7ca-2c93-4def-998b-85b716e00ed5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb5bf4323a274f72a3865aa7cc8dce4d', 'request_id': 'req-4673c7ca-2c93-4def-998b-85b716e00ed5'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:41.183839 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-4673c7ca-2c93-4def-998b-85b716e00ed5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb5bf4323a274f72a3865aa7cc8dce4d', 'request_id': 'req-4673c7ca-2c93-4def-998b-85b716e00ed5'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:41.184974 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-be0ecf08-f261-4526-9c5f-c4a19b7698b5 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:18:41.186231 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-be0ecf08-f261-4526-9c5f-c4a19b7698b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:41.206516 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-4673c7ca-2c93-4def-998b-85b716e00ed5 admin admin] The token cache is being invalidate because project cb5bf4323a274f72a3865aa7cc8dce4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 19:18:41.209022 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-4673c7ca-2c93-4def-998b-85b716e00ed5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:41.210855 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-4673c7ca-2c93-4def-998b-85b716e00ed5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:41.213265 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 865/1719] 10.222.0.61 () {58 vars in 1233 bytes} [Mon Aug 22 19:18:40 2022] DELETE /identity/v3/projects/cb5bf4323a274f72a3865aa7cc8dce4d => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:18:41.220305 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b06941-628f-40fe-a657-637b8cebadfa None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:41.220619 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b06941-628f-40fe-a657-637b8cebadfa None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:41.220893 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b06941-628f-40fe-a657-637b8cebadfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:41.292000 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-94b06941-628f-40fe-a657-637b8cebadfa None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:18:41.293836 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-94b06941-628f-40fe-a657-637b8cebadfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:41.329301 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:18:41.405340 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 855/1720] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:18:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:18:41.408901 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0ecf08-f261-4526-9c5f-c4a19b7698b5 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:41.440989 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0ecf08-f261-4526-9c5f-c4a19b7698b5 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:41.463530 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893803a1-c081-4047-9dde-e746f9b2f501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:41.477431 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893803a1-c081-4047-9dde-e746f9b2f501 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:41.478297 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893803a1-c081-4047-9dde-e746f9b2f501 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:41.480510 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893803a1-c081-4047-9dde-e746f9b2f501 admin admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:41.483139 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893803a1-c081-4047-9dde-e746f9b2f501 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:18:41.486131 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893803a1-c081-4047-9dde-e746f9b2f501 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:41.495937 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893803a1-c081-4047-9dde-e746f9b2f501 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:41.506333 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 866/1721] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:18:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:18:41.516096 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e57ba7b-4738-43fc-bb54-045f962a568b None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:41.516413 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e57ba7b-4738-43fc-bb54-045f962a568b None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:41.516660 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e57ba7b-4738-43fc-bb54-045f962a568b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:41.556117 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 856/1722] 10.222.0.61 () {58 vars in 1153 bytes} [Mon Aug 22 19:18:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:18:41.558211 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893803a1-c081-4047-9dde-e746f9b2f501 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:41.577940 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-6e57ba7b-4738-43fc-bb54-045f962a568b None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:18:41.579214 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-6e57ba7b-4738-43fc-bb54-045f962a568b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:41.585421 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893803a1-c081-4047-9dde-e746f9b2f501 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:41.613723 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 867/1723] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:18:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 19:18:41.618409 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e57ba7b-4738-43fc-bb54-045f962a568b None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:41.629861 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa81df0a-c9dd-4710-98f6-e7975625be1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:41.630518 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa81df0a-c9dd-4710-98f6-e7975625be1a admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:41.630798 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa81df0a-c9dd-4710-98f6-e7975625be1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:41.631071 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa81df0a-c9dd-4710-98f6-e7975625be1a admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:41.641379 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa81df0a-c9dd-4710-98f6-e7975625be1a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:41.643428 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa81df0a-c9dd-4710-98f6-e7975625be1a admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:41.649867 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e57ba7b-4738-43fc-bb54-045f962a568b None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:41.657542 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e01a10-3782-47dd-a84c-5ed4d7d1b132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:41.658177 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e01a10-3782-47dd-a84c-5ed4d7d1b132 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:41.658459 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e01a10-3782-47dd-a84c-5ed4d7d1b132 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:41.658755 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e01a10-3782-47dd-a84c-5ed4d7d1b132 admin admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:41.659419 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e01a10-3782-47dd-a84c-5ed4d7d1b132 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:18:41.659780 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e01a10-3782-47dd-a84c-5ed4d7d1b132 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:41.661399 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e01a10-3782-47dd-a84c-5ed4d7d1b132 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:41.693949 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 857/1724] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 19:18:41 2022] POST /identity/v3/projects => generated 365 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:18:41.698147 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa81df0a-c9dd-4710-98f6-e7975625be1a admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:41.710741 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 868/1725] 10.222.0.61 () {58 vars in 1153 bytes} [Mon Aug 22 19:18:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:18:41.715474 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e01a10-3782-47dd-a84c-5ed4d7d1b132 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:41.734499 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e01a10-3782-47dd-a84c-5ed4d7d1b132 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:41.894657 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7a5b02-bb53-45cd-a0c4-0ca47ef95757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:41.895344 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7a5b02-bb53-45cd-a0c4-0ca47ef95757 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:41.895450 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7a5b02-bb53-45cd-a0c4-0ca47ef95757 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:41.896163 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7a5b02-bb53-45cd-a0c4-0ca47ef95757 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:41.896449 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7a5b02-bb53-45cd-a0c4-0ca47ef95757 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:41.898382 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7a5b02-bb53-45cd-a0c4-0ca47ef95757 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:41.941546 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa81df0a-c9dd-4710-98f6-e7975625be1a admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:41.954263 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503dd723-f8b2-4920-ae81-4822b5fdf6dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:41.955017 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503dd723-f8b2-4920-ae81-4822b5fdf6dc admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:41.955517 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503dd723-f8b2-4920-ae81-4822b5fdf6dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:41.955888 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503dd723-f8b2-4920-ae81-4822b5fdf6dc admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:41.966201 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503dd723-f8b2-4920-ae81-4822b5fdf6dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:41.968693 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503dd723-f8b2-4920-ae81-4822b5fdf6dc admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:41.972457 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:18:41.992835 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 869/1726] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 19:18:41 2022] POST /identity/v3/projects => generated 363 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 19:18:41.993785 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7a5b02-bb53-45cd-a0c4-0ca47ef95757 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:42.013300 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-7d7a5b02-bb53-45cd-a0c4-0ca47ef95757 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:42.018714 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 858/1727] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 19:18:41 2022] POST /identity/v3/users => generated 283 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:18:42.020408 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503dd723-f8b2-4920-ae81-4822b5fdf6dc admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:42.049334 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503dd723-f8b2-4920-ae81-4822b5fdf6dc admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:42.258050 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077e47b9-c50d-4234-9bde-fb3b9f0f7a31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:42.259713 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077e47b9-c50d-4234-9bde-fb3b9f0f7a31 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:42.260621 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077e47b9-c50d-4234-9bde-fb3b9f0f7a31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:42.261476 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077e47b9-c50d-4234-9bde-fb3b9f0f7a31 admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:42.263276 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077e47b9-c50d-4234-9bde-fb3b9f0f7a31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:42.265274 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077e47b9-c50d-4234-9bde-fb3b9f0f7a31 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:42.300601 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7a5b02-bb53-45cd-a0c4-0ca47ef95757 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:42.310407 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5a8c47-1df9-4943-a5e5-d217061532a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:42.311106 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5a8c47-1df9-4943-a5e5-d217061532a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:42.311358 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5a8c47-1df9-4943-a5e5-d217061532a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:42.311637 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5a8c47-1df9-4943-a5e5-d217061532a3 admin admin] PATH_INFO: `/v3/projects/4c435019ac81431cb462873b6eaf636f` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:42.317461 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5a8c47-1df9-4943-a5e5-d217061532a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c435019ac81431cb462873b6eaf636f)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:42.319798 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5a8c47-1df9-4943-a5e5-d217061532a3 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:42.327603 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 859/1728] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 19:18:42 2022] POST /identity/v3/users => generated 282 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:18:42.328949 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077e47b9-c50d-4234-9bde-fb3b9f0f7a31 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:42.342069 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077e47b9-c50d-4234-9bde-fb3b9f0f7a31 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:42.348164 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea5fcda-bbbf-4bfa-ab45-1cc517b3a10b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:42.349974 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea5fcda-bbbf-4bfa-ab45-1cc517b3a10b admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:42.350441 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea5fcda-bbbf-4bfa-ab45-1cc517b3a10b admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:42.351166 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea5fcda-bbbf-4bfa-ab45-1cc517b3a10b admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:42.352451 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea5fcda-bbbf-4bfa-ab45-1cc517b3a10b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:42.354594 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea5fcda-bbbf-4bfa-ab45-1cc517b3a10b admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:42.372651 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 860/1729] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:18:42 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:18:42.376344 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940a11a2-00f4-40a3-ab3d-39c7269af545 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:42.377038 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940a11a2-00f4-40a3-ab3d-39c7269af545 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:42.377495 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940a11a2-00f4-40a3-ab3d-39c7269af545 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:42.379280 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 861/1730] 10.222.0.61 () {54 vars in 921 bytes} [Mon Aug 22 19:18:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:18:42.381319 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940a11a2-00f4-40a3-ab3d-39c7269af545 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:42.414331 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940a11a2-00f4-40a3-ab3d-39c7269af545 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:42.434938 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-aa5a8c47-1df9-4943-a5e5-d217061532a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c435019ac81431cb462873b6eaf636f', 'request_id': 'req-aa5a8c47-1df9-4943-a5e5-d217061532a3'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:42.438445 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:18:42.450758 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-aa5a8c47-1df9-4943-a5e5-d217061532a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c435019ac81431cb462873b6eaf636f', 'request_id': 'req-aa5a8c47-1df9-4943-a5e5-d217061532a3'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:42.460676 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-aa5a8c47-1df9-4943-a5e5-d217061532a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c435019ac81431cb462873b6eaf636f', 'request_id': 'req-aa5a8c47-1df9-4943-a5e5-d217061532a3'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:42.483481 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-aa5a8c47-1df9-4943-a5e5-d217061532a3 admin admin] The token cache is being invalidate because project 4c435019ac81431cb462873b6eaf636f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 19:18:42.483897 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-aa5a8c47-1df9-4943-a5e5-d217061532a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:42.492368 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-aa5a8c47-1df9-4943-a5e5-d217061532a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:42.493939 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 870/1731] 10.222.0.61 () {58 vars in 1233 bytes} [Mon Aug 22 19:18:41 2022] DELETE /identity/v3/projects/4c435019ac81431cb462873b6eaf636f => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:18:42.494997 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5a8c47-1df9-4943-a5e5-d217061532a3 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:42.496663 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-aa5a8c47-1df9-4943-a5e5-d217061532a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:42.518426 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33ab967-41de-49a9-bf32-7ec09ee63909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:42.519768 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33ab967-41de-49a9-bf32-7ec09ee63909 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:42.520051 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33ab967-41de-49a9-bf32-7ec09ee63909 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:42.520438 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33ab967-41de-49a9-bf32-7ec09ee63909 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:42.521213 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33ab967-41de-49a9-bf32-7ec09ee63909 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:42.524268 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33ab967-41de-49a9-bf32-7ec09ee63909 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:42.541347 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 862/1732] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:18:42 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:18:42.541347 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c144084b-0e67-4716-934d-06c29a2e54b0 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:42.541347 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c144084b-0e67-4716-934d-06c29a2e54b0 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:42.541347 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c144084b-0e67-4716-934d-06c29a2e54b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:42.577602 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-c144084b-0e67-4716-934d-06c29a2e54b0 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:18:42.594405 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-c144084b-0e67-4716-934d-06c29a2e54b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:42.616251 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 863/1733] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:18:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:18:42.619511 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c144084b-0e67-4716-934d-06c29a2e54b0 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:42.628877 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-c144084b-0e67-4716-934d-06c29a2e54b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:42.643753 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5a8c47-1df9-4943-a5e5-d217061532a3 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:42.649566 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3d7be6-3cd6-4536-a698-632cf55c668b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:42.650127 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3d7be6-3cd6-4536-a698-632cf55c668b admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:42.650410 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3d7be6-3cd6-4536-a698-632cf55c668b admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:42.650687 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3d7be6-3cd6-4536-a698-632cf55c668b admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:42.651349 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3d7be6-3cd6-4536-a698-632cf55c668b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:42.653058 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3d7be6-3cd6-4536-a698-632cf55c668b admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:42.656753 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c144084b-0e67-4716-934d-06c29a2e54b0 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:42.674438 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7fa188-e28b-42f7-9413-23eef947bed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:42.677188 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7fa188-e28b-42f7-9413-23eef947bed2 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:42.677528 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7fa188-e28b-42f7-9413-23eef947bed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:42.678590 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7fa188-e28b-42f7-9413-23eef947bed2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:42.679462 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7fa188-e28b-42f7-9413-23eef947bed2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:42.680929 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 871/1734] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:18:42 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:18:42.682622 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43a4486-b8d0-434f-8762-08e55ab1771d None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:42.683713 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43a4486-b8d0-434f-8762-08e55ab1771d None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:42.684595 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43a4486-b8d0-434f-8762-08e55ab1771d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:42.687984 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7fa188-e28b-42f7-9413-23eef947bed2 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:42.715826 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-e43a4486-b8d0-434f-8762-08e55ab1771d None None] MFA Rules not processed for user `e7f43e2756b746a6825ca1a06b7d7c91`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:18:42.718275 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-e43a4486-b8d0-434f-8762-08e55ab1771d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:42.738670 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 864/1735] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:18:42 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:18:42.738670 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8cb1c7-079d-4439-8c83-7d631592788f None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:42.738670 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8cb1c7-079d-4439-8c83-7d631592788f None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:42.738670 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8cb1c7-079d-4439-8c83-7d631592788f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:42.760874 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:18:42.837787 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 872/1736] 10.222.0.61 () {58 vars in 1018 bytes} [Mon Aug 22 19:18:42 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:18:42.838945 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43a4486-b8d0-434f-8762-08e55ab1771d None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:42.854226 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43a4486-b8d0-434f-8762-08e55ab1771d None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:42.860675 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfc5322-54ac-402a-b3ce-f7544f7b6411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:42.862308 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfc5322-54ac-402a-b3ce-f7544f7b6411 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:42.862808 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfc5322-54ac-402a-b3ce-f7544f7b6411 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:42.864139 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfc5322-54ac-402a-b3ce-f7544f7b6411 admin admin] PATH_INFO: `/v3/projects/4eb1fc6b915b4e2399d30bb3a11aa6ca/users/79aa5c4913bc4fe1802bf84bd1ec462e/roles/74a40bc819cb42eea7e966a4b2a016f1` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:42.865226 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-1d8cb1c7-079d-4439-8c83-7d631592788f None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:18:42.866572 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-1d8cb1c7-079d-4439-8c83-7d631592788f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:42.889071 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 865/1737] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:18:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:18:42.893215 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8cb1c7-079d-4439-8c83-7d631592788f None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:42.918858 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8cb1c7-079d-4439-8c83-7d631592788f None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:42.928958 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca1c44b-6967-4cb2-a6a8-61d146aa511b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:42.929728 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca1c44b-6967-4cb2-a6a8-61d146aa511b admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:42.929976 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca1c44b-6967-4cb2-a6a8-61d146aa511b admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:42.930250 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca1c44b-6967-4cb2-a6a8-61d146aa511b admin admin] PATH_INFO: `/v3/projects/53e36b333dc84f988f53422c76ba2946/users/6331c5ab05c14a00ac4053b6bab86650/roles/74a40bc819cb42eea7e966a4b2a016f1` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:42.940686 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfc5322-54ac-402a-b3ce-f7544f7b6411 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4eb1fc6b915b4e2399d30bb3a11aa6ca, user_id=79aa5c4913bc4fe1802bf84bd1ec462e, role_id=74a40bc819cb42eea7e966a4b2a016f1)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:42.943715 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfc5322-54ac-402a-b3ce-f7544f7b6411 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:42.966064 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 873/1738] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:18:42 2022] PUT /identity/v3/projects/4eb1fc6b915b4e2399d30bb3a11aa6ca/users/79aa5c4913bc4fe1802bf84bd1ec462e/roles/74a40bc819cb42eea7e966a4b2a016f1 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:18:42.967459 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566a3540-a9f4-4daa-95b5-c9b71b451a63 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:42.967810 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566a3540-a9f4-4daa-95b5-c9b71b451a63 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:42.967978 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566a3540-a9f4-4daa-95b5-c9b71b451a63 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:42.969100 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca1c44b-6967-4cb2-a6a8-61d146aa511b admin admin] RBAC: Authorizing `identity:create_grant(project_id=53e36b333dc84f988f53422c76ba2946, user_id=6331c5ab05c14a00ac4053b6bab86650, role_id=74a40bc819cb42eea7e966a4b2a016f1)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:42.969214 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 874/1739] 10.222.0.61 () {54 vars in 921 bytes} [Mon Aug 22 19:18:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:18:42.971843 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca1c44b-6967-4cb2-a6a8-61d146aa511b admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:42.972126 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566a3540-a9f4-4daa-95b5-c9b71b451a63 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:43.005777 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 866/1740] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:18:42 2022] PUT /identity/v3/projects/53e36b333dc84f988f53422c76ba2946/users/6331c5ab05c14a00ac4053b6bab86650/roles/74a40bc819cb42eea7e966a4b2a016f1 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:18:43.005777 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d0c83a-c171-4506-956d-c19320456405 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:43.005777 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d0c83a-c171-4506-956d-c19320456405 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:43.005777 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d0c83a-c171-4506-956d-c19320456405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:43.072295 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-67d0c83a-c171-4506-956d-c19320456405 None None] MFA Rules not processed for user `79aa5c4913bc4fe1802bf84bd1ec462e`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:18:43.073730 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-67d0c83a-c171-4506-956d-c19320456405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:43.147544 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566a3540-a9f4-4daa-95b5-c9b71b451a63 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:43.259389 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 867/1741] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:18:43 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:18:43.263467 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fc36df-51da-4b28-9aaa-b3469b95e89d None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:43.264263 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fc36df-51da-4b28-9aaa-b3469b95e89d None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:43.264548 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fc36df-51da-4b28-9aaa-b3469b95e89d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:43.297131 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb10e79-dc29-4868-b274-f7ac54906b1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:43.306073 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb10e79-dc29-4868-b274-f7ac54906b1c admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:43.306510 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb10e79-dc29-4868-b274-f7ac54906b1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:43.307281 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb10e79-dc29-4868-b274-f7ac54906b1c admin admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:43.309669 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb10e79-dc29-4868-b274-f7ac54906b1c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:18:43.310105 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb10e79-dc29-4868-b274-f7ac54906b1c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:43.314938 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb10e79-dc29-4868-b274-f7ac54906b1c admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:43.354465 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 875/1742] 10.222.0.61 () {58 vars in 1153 bytes} [Mon Aug 22 19:18:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 385 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:18:43.359683 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb10e79-dc29-4868-b274-f7ac54906b1c admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:43.368408 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-5fb10e79-dc29-4868-b274-f7ac54906b1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:43.381665 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-21fc36df-51da-4b28-9aaa-b3469b95e89d None None] MFA Rules not processed for user `6331c5ab05c14a00ac4053b6bab86650`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:18:43.399250 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-21fc36df-51da-4b28-9aaa-b3469b95e89d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:43.436669 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:18:43.495147 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb10e79-dc29-4868-b274-f7ac54906b1c admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:43.502633 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83701ca5-cba5-4ca0-81f8-35c6a070d4c0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f43e2756b746a6825ca1a06b7d7c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:43.504270 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83701ca5-cba5-4ca0-81f8-35c6a070d4c0 service nova] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:43.504270 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83701ca5-cba5-4ca0-81f8-35c6a070d4c0 service nova] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:43.504270 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83701ca5-cba5-4ca0-81f8-35c6a070d4c0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:43.522082 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:18:43.522718 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83701ca5-cba5-4ca0-81f8-35c6a070d4c0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:43.524030 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83701ca5-cba5-4ca0-81f8-35c6a070d4c0 service nova] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:43.575257 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 876/1743] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:18:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:18:43.587391 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff53a55-b86c-432c-a3f3-d1735c1d0bb0 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:43.588526 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff53a55-b86c-432c-a3f3-d1735c1d0bb0 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:43.589958 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff53a55-b86c-432c-a3f3-d1735c1d0bb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:43.604464 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 868/1744] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:18:43 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 344 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:18:43.605837 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fc36df-51da-4b28-9aaa-b3469b95e89d None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:43.648122 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fc36df-51da-4b28-9aaa-b3469b95e89d None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:43.656897 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25643b1-2496-404c-a2c7-4ef0e57d4c12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:43.657630 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25643b1-2496-404c-a2c7-4ef0e57d4c12 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:43.657955 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25643b1-2496-404c-a2c7-4ef0e57d4c12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:43.658432 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25643b1-2496-404c-a2c7-4ef0e57d4c12 admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:43.659364 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25643b1-2496-404c-a2c7-4ef0e57d4c12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:43.664429 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25643b1-2496-404c-a2c7-4ef0e57d4c12 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:43.672538 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-cff53a55-b86c-432c-a3f3-d1735c1d0bb0 None None] MFA Rules not processed for user `e7f43e2756b746a6825ca1a06b7d7c91`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:18:43.676861 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-cff53a55-b86c-432c-a3f3-d1735c1d0bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:43.708516 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 877/1745] 10.222.0.61 () {58 vars in 1018 bytes} [Mon Aug 22 19:18:43 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:18:43.711583 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff53a55-b86c-432c-a3f3-d1735c1d0bb0 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:43.725589 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 869/1746] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 19:18:43 2022] POST /identity/v3/projects => generated 365 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:18:43.727198 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25643b1-2496-404c-a2c7-4ef0e57d4c12 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:43.740802 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff53a55-b86c-432c-a3f3-d1735c1d0bb0 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:43.878254 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25643b1-2496-404c-a2c7-4ef0e57d4c12 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:43.900721 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1533169e-2de4-4829-827a-0aae19b01a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:43.902478 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1533169e-2de4-4829-827a-0aae19b01a8c admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:43.903260 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1533169e-2de4-4829-827a-0aae19b01a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:43.904453 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1533169e-2de4-4829-827a-0aae19b01a8c admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:43.905302 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1533169e-2de4-4829-827a-0aae19b01a8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:43.906207 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae688ed-0d7d-4a8c-8322-e468cdc6fad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:43.908250 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae688ed-0d7d-4a8c-8322-e468cdc6fad3 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:43.909763 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae688ed-0d7d-4a8c-8322-e468cdc6fad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:43.910710 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1533169e-2de4-4829-827a-0aae19b01a8c admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:43.911138 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae688ed-0d7d-4a8c-8322-e468cdc6fad3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:43.915614 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae688ed-0d7d-4a8c-8322-e468cdc6fad3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:43.917734 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae688ed-0d7d-4a8c-8322-e468cdc6fad3 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:43.972935 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 870/1747] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 19:18:43 2022] POST /identity/v3/projects => generated 363 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:18:43.975402 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1533169e-2de4-4829-827a-0aae19b01a8c admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:43.981187 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 878/1748] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 19:18:43 2022] POST /identity/v3/projects => generated 377 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:18:43.981187 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae688ed-0d7d-4a8c-8322-e468cdc6fad3 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:43.985147 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-fae688ed-0d7d-4a8c-8322-e468cdc6fad3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:44.230863 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae688ed-0d7d-4a8c-8322-e468cdc6fad3 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:44.242889 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1533169e-2de4-4829-827a-0aae19b01a8c admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:44.249868 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a293cb-9c3d-47ea-b7cd-5330f5ddb0ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f43e2756b746a6825ca1a06b7d7c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:44.255753 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a293cb-9c3d-47ea-b7cd-5330f5ddb0ba service nova] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:44.258867 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a293cb-9c3d-47ea-b7cd-5330f5ddb0ba service nova] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:44.259435 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a293cb-9c3d-47ea-b7cd-5330f5ddb0ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:44.295328 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a293cb-9c3d-47ea-b7cd-5330f5ddb0ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:44.297378 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a293cb-9c3d-47ea-b7cd-5330f5ddb0ba service nova] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:44.383310 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:18:44.411874 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 879/1749] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:18:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:18:44.415457 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a293cb-9c3d-47ea-b7cd-5330f5ddb0ba service nova] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:44.443183 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04697f45-8eb5-42c2-9465-e4c65448fcdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:44.448066 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04697f45-8eb5-42c2-9465-e4c65448fcdc admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:44.448388 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04697f45-8eb5-42c2-9465-e4c65448fcdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:44.449207 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04697f45-8eb5-42c2-9465-e4c65448fcdc admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:44.449946 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04697f45-8eb5-42c2-9465-e4c65448fcdc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:44.452010 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04697f45-8eb5-42c2-9465-e4c65448fcdc admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:44.458325 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a293cb-9c3d-47ea-b7cd-5330f5ddb0ba service nova] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:44.490864 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727e3e6c-ded8-4811-8352-ec1b4646f17d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:44.494081 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727e3e6c-ded8-4811-8352-ec1b4646f17d admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:44.495563 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727e3e6c-ded8-4811-8352-ec1b4646f17d admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:44.495563 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727e3e6c-ded8-4811-8352-ec1b4646f17d admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:44.499696 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727e3e6c-ded8-4811-8352-ec1b4646f17d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:44.505540 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727e3e6c-ded8-4811-8352-ec1b4646f17d admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:44.508606 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 871/1750] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 19:18:43 2022] POST /identity/v3/users => generated 289 bytes in 535 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:18:44.509566 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04697f45-8eb5-42c2-9465-e4c65448fcdc admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:44.535280 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04697f45-8eb5-42c2-9465-e4c65448fcdc admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:44.549803 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844bba5e-0cfa-4626-b9c5-ffeba7b06c8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:44.550672 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844bba5e-0cfa-4626-b9c5-ffeba7b06c8d admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:44.551142 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844bba5e-0cfa-4626-b9c5-ffeba7b06c8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:44.551602 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844bba5e-0cfa-4626-b9c5-ffeba7b06c8d admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:44.552466 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844bba5e-0cfa-4626-b9c5-ffeba7b06c8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:44.554600 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844bba5e-0cfa-4626-b9c5-ffeba7b06c8d admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:44.597711 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 872/1751] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 19:18:44 2022] POST /identity/v3/users => generated 289 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:18:44.601472 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844bba5e-0cfa-4626-b9c5-ffeba7b06c8d admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:44.603470 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 880/1752] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 19:18:44 2022] POST /identity/v3/users => generated 288 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:18:44.604471 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727e3e6c-ded8-4811-8352-ec1b4646f17d admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:44.622954 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:18:44.639977 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727e3e6c-ded8-4811-8352-ec1b4646f17d admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:44.642023 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844bba5e-0cfa-4626-b9c5-ffeba7b06c8d admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:44.653313 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce582b8-943c-44a6-ac5c-2f7f17567057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:44.653954 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce582b8-943c-44a6-ac5c-2f7f17567057 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:44.654228 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce582b8-943c-44a6-ac5c-2f7f17567057 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:44.654495 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce582b8-943c-44a6-ac5c-2f7f17567057 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:44.655148 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce582b8-943c-44a6-ac5c-2f7f17567057 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:44.661793 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be1513f-3445-4682-b0f7-0c0636da6d12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:44.662638 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be1513f-3445-4682-b0f7-0c0636da6d12 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:44.663107 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be1513f-3445-4682-b0f7-0c0636da6d12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:44.663841 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be1513f-3445-4682-b0f7-0c0636da6d12 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:44.664684 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be1513f-3445-4682-b0f7-0c0636da6d12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:44.667365 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be1513f-3445-4682-b0f7-0c0636da6d12 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:44.670942 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce582b8-943c-44a6-ac5c-2f7f17567057 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:44.686194 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 873/1753] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:18:44 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:18:44.691402 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be1513f-3445-4682-b0f7-0c0636da6d12 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:44.706774 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be1513f-3445-4682-b0f7-0c0636da6d12 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:44.713049 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2c3cae-f4f2-4791-8bee-228f1c37e550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:44.713858 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2c3cae-f4f2-4791-8bee-228f1c37e550 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:44.714929 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2c3cae-f4f2-4791-8bee-228f1c37e550 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:44.715302 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2c3cae-f4f2-4791-8bee-228f1c37e550 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:44.716100 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2c3cae-f4f2-4791-8bee-228f1c37e550 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:44.718364 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2c3cae-f4f2-4791-8bee-228f1c37e550 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:44.722672 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 881/1754] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:18:44 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:18:44.723969 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce582b8-943c-44a6-ac5c-2f7f17567057 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:44.737170 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 874/1755] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:18:44 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:18:44.738477 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2c3cae-f4f2-4791-8bee-228f1c37e550 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:44.754679 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce582b8-943c-44a6-ac5c-2f7f17567057 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:44.768454 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cba0285-0479-409a-a2be-85589565f474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:44.769441 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2c3cae-f4f2-4791-8bee-228f1c37e550 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:44.769585 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cba0285-0479-409a-a2be-85589565f474 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:44.769779 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cba0285-0479-409a-a2be-85589565f474 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:44.770057 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cba0285-0479-409a-a2be-85589565f474 admin admin] PATH_INFO: `/v3/projects/80cba7f2f9f448dc8e3f0a33d8e88a9f/users/b1dfd418a9cb4e35b5be038a8b0a1b57/roles/163c30c3b3c246beb849ce176f02d9da` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:44.782668 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f282b3-289f-4383-aa91-f9383165373d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:44.783317 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f282b3-289f-4383-aa91-f9383165373d admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:44.783588 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f282b3-289f-4383-aa91-f9383165373d admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:44.783892 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f282b3-289f-4383-aa91-f9383165373d admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:44.794767 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cba0285-0479-409a-a2be-85589565f474 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80cba7f2f9f448dc8e3f0a33d8e88a9f, user_id=b1dfd418a9cb4e35b5be038a8b0a1b57, role_id=163c30c3b3c246beb849ce176f02d9da)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:44.797490 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cba0285-0479-409a-a2be-85589565f474 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:44.803287 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f282b3-289f-4383-aa91-f9383165373d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:44.809495 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f282b3-289f-4383-aa91-f9383165373d admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:44.824617 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 882/1756] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:18:44 2022] PUT /identity/v3/projects/80cba7f2f9f448dc8e3f0a33d8e88a9f/users/b1dfd418a9cb4e35b5be038a8b0a1b57/roles/163c30c3b3c246beb849ce176f02d9da => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:18:44.826017 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cba0285-0479-409a-a2be-85589565f474 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:44.846749 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 875/1757] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:18:44 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:18:44.846749 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f282b3-289f-4383-aa91-f9383165373d admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:44.866749 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f282b3-289f-4383-aa91-f9383165373d admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:45.023482 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c903ca5-0e0e-4e9f-b54f-d5909013d677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:45.024829 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c903ca5-0e0e-4e9f-b54f-d5909013d677 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:45.026221 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c903ca5-0e0e-4e9f-b54f-d5909013d677 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:45.028761 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c903ca5-0e0e-4e9f-b54f-d5909013d677 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:45.032021 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c903ca5-0e0e-4e9f-b54f-d5909013d677 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:45.035097 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c903ca5-0e0e-4e9f-b54f-d5909013d677 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:45.084206 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 876/1758] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:18:44 2022] GET /identity/v3/roles => generated 1420 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:18:45.090057 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c903ca5-0e0e-4e9f-b54f-d5909013d677 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:45.116927 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cba0285-0479-409a-a2be-85589565f474 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:45.125384 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c903ca5-0e0e-4e9f-b54f-d5909013d677 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:45.132222 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2f6d08-4073-4cad-91b4-dafd26220177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:45.133230 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2f6d08-4073-4cad-91b4-dafd26220177 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:45.133714 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2f6d08-4073-4cad-91b4-dafd26220177 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:45.134184 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2f6d08-4073-4cad-91b4-dafd26220177 admin admin] PATH_INFO: `/v3/projects/a0189ca307584838bf7ede9fb4843070/users/b21ad8c810474c67a82f6a285ba1664e/roles/163c30c3b3c246beb849ce176f02d9da` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:45.149713 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa87dd43-4780-4d1d-bcc4-030db93cd700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:45.150466 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa87dd43-4780-4d1d-bcc4-030db93cd700 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:45.150690 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa87dd43-4780-4d1d-bcc4-030db93cd700 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:45.150983 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa87dd43-4780-4d1d-bcc4-030db93cd700 admin admin] PATH_INFO: `/v3/projects/a15343703c7144bf8677126a9ada39da/users/3bfd09479ca64c3f86d0d73763e29464/roles/74a40bc819cb42eea7e966a4b2a016f1` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:45.178529 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2f6d08-4073-4cad-91b4-dafd26220177 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0189ca307584838bf7ede9fb4843070, user_id=b21ad8c810474c67a82f6a285ba1664e, role_id=163c30c3b3c246beb849ce176f02d9da)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:45.181618 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2f6d08-4073-4cad-91b4-dafd26220177 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:45.193545 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa87dd43-4780-4d1d-bcc4-030db93cd700 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a15343703c7144bf8677126a9ada39da, user_id=3bfd09479ca64c3f86d0d73763e29464, role_id=74a40bc819cb42eea7e966a4b2a016f1)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:45.199277 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa87dd43-4780-4d1d-bcc4-030db93cd700 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:45.211756 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 883/1759] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:18:44 2022] PUT /identity/v3/projects/a0189ca307584838bf7ede9fb4843070/users/b21ad8c810474c67a82f6a285ba1664e/roles/163c30c3b3c246beb849ce176f02d9da => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:18:45.213437 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2f6d08-4073-4cad-91b4-dafd26220177 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:45.259045 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 877/1760] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:18:45 2022] PUT /identity/v3/projects/a15343703c7144bf8677126a9ada39da/users/3bfd09479ca64c3f86d0d73763e29464/roles/74a40bc819cb42eea7e966a4b2a016f1 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:18:45.260112 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa87dd43-4780-4d1d-bcc4-030db93cd700 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:45.396898 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2f6d08-4073-4cad-91b4-dafd26220177 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:45.404938 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5724e60-82e6-4507-a880-3b12d81bfb46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:45.405149 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa87dd43-4780-4d1d-bcc4-030db93cd700 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:45.405719 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5724e60-82e6-4507-a880-3b12d81bfb46 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:45.406081 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5724e60-82e6-4507-a880-3b12d81bfb46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:45.406669 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5724e60-82e6-4507-a880-3b12d81bfb46 admin admin] PATH_INFO: `/v3/projects/80cba7f2f9f448dc8e3f0a33d8e88a9f/users/b1dfd418a9cb4e35b5be038a8b0a1b57/roles/163c30c3b3c246beb849ce176f02d9da` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:45.408926 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5724e60-82e6-4507-a880-3b12d81bfb46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80cba7f2f9f448dc8e3f0a33d8e88a9f, user_id=b1dfd418a9cb4e35b5be038a8b0a1b57, role_id=163c30c3b3c246beb849ce176f02d9da)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:45.412876 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5724e60-82e6-4507-a880-3b12d81bfb46 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:45.424044 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b253431-0381-4f3b-824a-1436d3594cf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:45.424697 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b253431-0381-4f3b-824a-1436d3594cf0 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:45.424997 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b253431-0381-4f3b-824a-1436d3594cf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:45.425306 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b253431-0381-4f3b-824a-1436d3594cf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:45.425941 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b253431-0381-4f3b-824a-1436d3594cf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:45.427674 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b253431-0381-4f3b-824a-1436d3594cf0 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:45.441657 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 884/1761] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:18:45 2022] PUT /identity/v3/projects/80cba7f2f9f448dc8e3f0a33d8e88a9f/users/b1dfd418a9cb4e35b5be038a8b0a1b57/roles/163c30c3b3c246beb849ce176f02d9da => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:18:45.445963 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb28ee10-b146-4f52-bb03-6a3d3c52f52f None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:45.447342 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb28ee10-b146-4f52-bb03-6a3d3c52f52f None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:45.447968 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb28ee10-b146-4f52-bb03-6a3d3c52f52f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:45.460753 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 878/1762] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:18:45 2022] GET /identity/v3/roles => generated 1420 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:18:45.462195 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524aa7d2-64b0-446f-92d5-82cc9a3a682b None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:45.462492 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524aa7d2-64b0-446f-92d5-82cc9a3a682b None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:45.462753 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524aa7d2-64b0-446f-92d5-82cc9a3a682b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:45.483554 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:18:45.504494 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-cb28ee10-b146-4f52-bb03-6a3d3c52f52f None None] MFA Rules not processed for user `3bfd09479ca64c3f86d0d73763e29464`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:18:45.506403 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-cb28ee10-b146-4f52-bb03-6a3d3c52f52f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:45.534532 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-524aa7d2-64b0-446f-92d5-82cc9a3a682b None None] MFA Rules not processed for user `b1dfd418a9cb4e35b5be038a8b0a1b57`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:18:45.535820 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-524aa7d2-64b0-446f-92d5-82cc9a3a682b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:45.555201 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:18:45.667970 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 885/1763] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:18:45 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:18:45.669328 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb28ee10-b146-4f52-bb03-6a3d3c52f52f None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:45.704456 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 879/1764] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:18:45 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:18:45.713975 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524aa7d2-64b0-446f-92d5-82cc9a3a682b None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:45.816602 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb28ee10-b146-4f52-bb03-6a3d3c52f52f None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:45.824678 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af38b98-444d-4da3-9ea2-8d4a16bb21f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:45.825717 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af38b98-444d-4da3-9ea2-8d4a16bb21f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:45.826643 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af38b98-444d-4da3-9ea2-8d4a16bb21f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:45.827074 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af38b98-444d-4da3-9ea2-8d4a16bb21f8 admin admin] PATH_INFO: `/v3/projects/a0189ca307584838bf7ede9fb4843070/users/b21ad8c810474c67a82f6a285ba1664e/roles/163c30c3b3c246beb849ce176f02d9da` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:45.832134 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af38b98-444d-4da3-9ea2-8d4a16bb21f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0189ca307584838bf7ede9fb4843070, user_id=b21ad8c810474c67a82f6a285ba1664e, role_id=163c30c3b3c246beb849ce176f02d9da)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:45.835441 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af38b98-444d-4da3-9ea2-8d4a16bb21f8 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:45.856112 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524aa7d2-64b0-446f-92d5-82cc9a3a682b None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:45.868353 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3782592-a496-4069-9c74-1d6cc10b1a7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:45.868910 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3782592-a496-4069-9c74-1d6cc10b1a7d admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:45.869193 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3782592-a496-4069-9c74-1d6cc10b1a7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:45.869466 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3782592-a496-4069-9c74-1d6cc10b1a7d admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:45.870127 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3782592-a496-4069-9c74-1d6cc10b1a7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:45.872126 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3782592-a496-4069-9c74-1d6cc10b1a7d admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:45.873111 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 886/1765] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:18:45 2022] PUT /identity/v3/projects/a0189ca307584838bf7ede9fb4843070/users/b21ad8c810474c67a82f6a285ba1664e/roles/163c30c3b3c246beb849ce176f02d9da => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:18:45.874016 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af38b98-444d-4da3-9ea2-8d4a16bb21f8 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:45.883045 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-4af38b98-444d-4da3-9ea2-8d4a16bb21f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:45.952423 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 880/1766] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 19:18:45 2022] POST /identity/v3/projects => generated 377 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:18:45.953927 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffd7e5a-2834-4a51-9985-07141b2eaf3f None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:45.954359 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffd7e5a-2834-4a51-9985-07141b2eaf3f None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:45.954865 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffd7e5a-2834-4a51-9985-07141b2eaf3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:46.000855 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-dffd7e5a-2834-4a51-9985-07141b2eaf3f None None] MFA Rules not processed for user `b21ad8c810474c67a82f6a285ba1664e`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:18:46.002401 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-dffd7e5a-2834-4a51-9985-07141b2eaf3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:46.014388 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af38b98-444d-4da3-9ea2-8d4a16bb21f8 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:46.159506 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:18:46.179254 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 881/1767] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:18:45 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:18:46.180637 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffd7e5a-2834-4a51-9985-07141b2eaf3f None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:46.223460 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07eb3df-cf4c-4eed-8074-79d6e94f0f90 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e79498873d4ed29a0e19bbea9b9214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:46.232538 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07eb3df-cf4c-4eed-8074-79d6e94f0f90 service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:46.232830 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07eb3df-cf4c-4eed-8074-79d6e94f0f90 service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:46.233091 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07eb3df-cf4c-4eed-8074-79d6e94f0f90 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:46.300985 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:18:46.310814 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffd7e5a-2834-4a51-9985-07141b2eaf3f None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:46.317142 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dc2f60-9b48-4e1f-95ff-1d0253690f64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:46.318274 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dc2f60-9b48-4e1f-95ff-1d0253690f64 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:46.318647 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dc2f60-9b48-4e1f-95ff-1d0253690f64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:46.318921 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dc2f60-9b48-4e1f-95ff-1d0253690f64 admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:46.319997 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dc2f60-9b48-4e1f-95ff-1d0253690f64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:46.321966 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dc2f60-9b48-4e1f-95ff-1d0253690f64 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:46.345644 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07eb3df-cf4c-4eed-8074-79d6e94f0f90 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:46.347966 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07eb3df-cf4c-4eed-8074-79d6e94f0f90 service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:46.352016 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 882/1768] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 19:18:46 2022] POST /identity/v3/users => generated 295 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:18:46.352564 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dc2f60-9b48-4e1f-95ff-1d0253690f64 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:46.357703 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-21dc2f60-9b48-4e1f-95ff-1d0253690f64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:46.364687 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 887/1769] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:18:45 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:18:46.365369 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07eb3df-cf4c-4eed-8074-79d6e94f0f90 service cinder] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:46.379625 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07eb3df-cf4c-4eed-8074-79d6e94f0f90 service cinder] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:46.384506 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47635670-fec4-4d07-8a24-d8d724851fa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:46.385453 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47635670-fec4-4d07-8a24-d8d724851fa1 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:46.386723 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47635670-fec4-4d07-8a24-d8d724851fa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:46.386723 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47635670-fec4-4d07-8a24-d8d724851fa1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:46.387613 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47635670-fec4-4d07-8a24-d8d724851fa1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:46.389520 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47635670-fec4-4d07-8a24-d8d724851fa1 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:46.395084 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dc2f60-9b48-4e1f-95ff-1d0253690f64 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:46.400708 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91af87af-0e39-4de0-9fb2-9adca73c0312 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e79498873d4ed29a0e19bbea9b9214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:46.401387 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91af87af-0e39-4de0-9fb2-9adca73c0312 service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:46.403689 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 888/1770] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:18:46 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:18:46.403840 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91af87af-0e39-4de0-9fb2-9adca73c0312 service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:46.403840 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91af87af-0e39-4de0-9fb2-9adca73c0312 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:46.406272 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47635670-fec4-4d07-8a24-d8d724851fa1 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:46.418878 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91af87af-0e39-4de0-9fb2-9adca73c0312 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:46.418878 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91af87af-0e39-4de0-9fb2-9adca73c0312 service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:46.425659 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47635670-fec4-4d07-8a24-d8d724851fa1 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:46.432177 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 883/1771] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:18:46 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:18:46.435090 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1f102a-7773-402c-bbb6-52fdf5b0a006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:46.436022 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1f102a-7773-402c-bbb6-52fdf5b0a006 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:46.436483 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1f102a-7773-402c-bbb6-52fdf5b0a006 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:46.436881 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1f102a-7773-402c-bbb6-52fdf5b0a006 admin admin] PATH_INFO: `/v3/projects/dd36ddc018e34297844281df4a35239c/users/8b85da6d1d414ff3ae823a8e307b2321/roles/163c30c3b3c246beb849ce176f02d9da` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:46.460247 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1f102a-7773-402c-bbb6-52fdf5b0a006 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd36ddc018e34297844281df4a35239c, user_id=8b85da6d1d414ff3ae823a8e307b2321, role_id=163c30c3b3c246beb849ce176f02d9da)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:46.464198 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1f102a-7773-402c-bbb6-52fdf5b0a006 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:46.493886 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 889/1772] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:18:46 2022] PUT /identity/v3/projects/dd36ddc018e34297844281df4a35239c/users/8b85da6d1d414ff3ae823a8e307b2321/roles/163c30c3b3c246beb849ce176f02d9da => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:18:46.497053 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91af87af-0e39-4de0-9fb2-9adca73c0312 service cinder] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:46.507875 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91af87af-0e39-4de0-9fb2-9adca73c0312 service cinder] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:46.535211 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1f102a-7773-402c-bbb6-52fdf5b0a006 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:46.603590 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1f102a-7773-402c-bbb6-52fdf5b0a006 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:46.609228 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31bf97d-4753-4bdc-92b2-7d2377b3002c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e79498873d4ed29a0e19bbea9b9214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:46.609874 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a041b3-8844-4ceb-846d-2bb20bdc6101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:46.610401 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31bf97d-4753-4bdc-92b2-7d2377b3002c service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:46.610401 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31bf97d-4753-4bdc-92b2-7d2377b3002c service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:46.610540 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a041b3-8844-4ceb-846d-2bb20bdc6101 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:46.610637 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31bf97d-4753-4bdc-92b2-7d2377b3002c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:46.610788 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a041b3-8844-4ceb-846d-2bb20bdc6101 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:46.611036 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a041b3-8844-4ceb-846d-2bb20bdc6101 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:46.611730 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a041b3-8844-4ceb-846d-2bb20bdc6101 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:46.613811 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a041b3-8844-4ceb-846d-2bb20bdc6101 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:46.638790 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 884/1773] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:18:46 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:18:46.642730 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a041b3-8844-4ceb-846d-2bb20bdc6101 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:46.657512 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a041b3-8844-4ceb-846d-2bb20bdc6101 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:46.686803 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a464defb-4352-4e1b-a07f-ca6406c79517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:46.687548 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a464defb-4352-4e1b-a07f-ca6406c79517 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:46.687923 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a464defb-4352-4e1b-a07f-ca6406c79517 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:46.688156 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a464defb-4352-4e1b-a07f-ca6406c79517 admin admin] PATH_INFO: `/v3/projects/dd36ddc018e34297844281df4a35239c/users/8b85da6d1d414ff3ae823a8e307b2321/roles/163c30c3b3c246beb849ce176f02d9da` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:46.691543 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a464defb-4352-4e1b-a07f-ca6406c79517 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd36ddc018e34297844281df4a35239c, user_id=8b85da6d1d414ff3ae823a8e307b2321, role_id=163c30c3b3c246beb849ce176f02d9da)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:46.699216 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a464defb-4352-4e1b-a07f-ca6406c79517 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:46.706598 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31bf97d-4753-4bdc-92b2-7d2377b3002c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:46.709133 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31bf97d-4753-4bdc-92b2-7d2377b3002c service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:46.723063 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 885/1774] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:18:46 2022] PUT /identity/v3/projects/dd36ddc018e34297844281df4a35239c/users/8b85da6d1d414ff3ae823a8e307b2321/roles/163c30c3b3c246beb849ce176f02d9da => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 19:18:46.730762 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 890/1775] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:18:46 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 19:18:46.734384 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cce7997-c63f-45b6-b586-9d485b8bcb91 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:46.734731 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cce7997-c63f-45b6-b586-9d485b8bcb91 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:46.735608 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cce7997-c63f-45b6-b586-9d485b8bcb91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:46.772093 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-0cce7997-c63f-45b6-b586-9d485b8bcb91 None None] MFA Rules not processed for user `8b85da6d1d414ff3ae823a8e307b2321`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:18:46.774359 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-0cce7997-c63f-45b6-b586-9d485b8bcb91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:46.856458 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:18:46.909004 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 886/1776] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:18:46 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:18:46.921708 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31bf97d-4753-4bdc-92b2-7d2377b3002c service cinder] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:46.934729 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31bf97d-4753-4bdc-92b2-7d2377b3002c service cinder] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:47.004525 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c2999b-ed16-446a-9190-9af78149f703 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e79498873d4ed29a0e19bbea9b9214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:47.005188 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c2999b-ed16-446a-9190-9af78149f703 service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:47.005677 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c2999b-ed16-446a-9190-9af78149f703 service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:47.005677 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c2999b-ed16-446a-9190-9af78149f703 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:47.016971 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:18:47.046105 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cce7997-c63f-45b6-b586-9d485b8bcb91 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:47.102825 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c2999b-ed16-446a-9190-9af78149f703 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:47.104526 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c2999b-ed16-446a-9190-9af78149f703 service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:47.120135 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 891/1777] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:18:46 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:18:47.144751 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cce7997-c63f-45b6-b586-9d485b8bcb91 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:47.150835 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7265018-e2a4-4dc3-a923-6f36966e1411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:47.152281 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7265018-e2a4-4dc3-a923-6f36966e1411 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:47.152568 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7265018-e2a4-4dc3-a923-6f36966e1411 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:47.152847 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7265018-e2a4-4dc3-a923-6f36966e1411 admin admin] PATH_INFO: `/v3/users/79aa5c4913bc4fe1802bf84bd1ec462e` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:47.154812 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7265018-e2a4-4dc3-a923-6f36966e1411 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79aa5c4913bc4fe1802bf84bd1ec462e)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:47.157850 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7265018-e2a4-4dc3-a923-6f36966e1411 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:47.230287 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-b7265018-e2a4-4dc3-a923-6f36966e1411 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79aa5c4913bc4fe1802bf84bd1ec462e'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:47.231135 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-b7265018-e2a4-4dc3-a923-6f36966e1411 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79aa5c4913bc4fe1802bf84bd1ec462e'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:47.239669 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-b7265018-e2a4-4dc3-a923-6f36966e1411 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79aa5c4913bc4fe1802bf84bd1ec462e'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:47.255486 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-b7265018-e2a4-4dc3-a923-6f36966e1411 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79aa5c4913bc4fe1802bf84bd1ec462e'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:47.270689 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-b7265018-e2a4-4dc3-a923-6f36966e1411 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79aa5c4913bc4fe1802bf84bd1ec462e'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:47.284352 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ecb2c7-6734-47d2-8387-16fb522a187b None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:47.284889 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ecb2c7-6734-47d2-8387-16fb522a187b None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:47.284889 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ecb2c7-6734-47d2-8387-16fb522a187b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:47.289450 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 887/1778] 10.222.0.61 () {58 vars in 1224 bytes} [Mon Aug 22 19:18:47 2022] DELETE /identity/v3/users/79aa5c4913bc4fe1802bf84bd1ec462e => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:18:47.313659 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-70ecb2c7-6734-47d2-8387-16fb522a187b None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:18:47.315080 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-70ecb2c7-6734-47d2-8387-16fb522a187b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:47.416535 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 892/1779] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:18:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:18:47.427978 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7265018-e2a4-4dc3-a923-6f36966e1411 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:47.429871 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-b7265018-e2a4-4dc3-a923-6f36966e1411 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:47.510369 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7265018-e2a4-4dc3-a923-6f36966e1411 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:47.514812 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c4e448-0593-4acc-9a42-ba726d38b1da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:47.515787 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c4e448-0593-4acc-9a42-ba726d38b1da service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:47.515881 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c4e448-0593-4acc-9a42-ba726d38b1da service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:47.516202 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c4e448-0593-4acc-9a42-ba726d38b1da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:47.526148 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c4e448-0593-4acc-9a42-ba726d38b1da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:47.528325 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c4e448-0593-4acc-9a42-ba726d38b1da service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:47.538966 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 888/1780] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:18:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 19:18:47.817536 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ecb2c7-6734-47d2-8387-16fb522a187b None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:47.819070 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-70ecb2c7-6734-47d2-8387-16fb522a187b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:47.833092 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ecb2c7-6734-47d2-8387-16fb522a187b None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:47.846972 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d93e475-75e1-4e6f-a8cd-d51e8cc9d0a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:47.847788 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d93e475-75e1-4e6f-a8cd-d51e8cc9d0a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:47.848133 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d93e475-75e1-4e6f-a8cd-d51e8cc9d0a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:47.848325 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d93e475-75e1-4e6f-a8cd-d51e8cc9d0a9 admin admin] PATH_INFO: `/v3/projects/4eb1fc6b915b4e2399d30bb3a11aa6ca` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:47.849469 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d93e475-75e1-4e6f-a8cd-d51e8cc9d0a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4eb1fc6b915b4e2399d30bb3a11aa6ca)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:47.851366 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d93e475-75e1-4e6f-a8cd-d51e8cc9d0a9 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:47.898114 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-5d93e475-75e1-4e6f-a8cd-d51e8cc9d0a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4eb1fc6b915b4e2399d30bb3a11aa6ca', 'request_id': 'req-5d93e475-75e1-4e6f-a8cd-d51e8cc9d0a9'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:47.905726 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-5d93e475-75e1-4e6f-a8cd-d51e8cc9d0a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4eb1fc6b915b4e2399d30bb3a11aa6ca', 'request_id': 'req-5d93e475-75e1-4e6f-a8cd-d51e8cc9d0a9'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:47.912065 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-5d93e475-75e1-4e6f-a8cd-d51e8cc9d0a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4eb1fc6b915b4e2399d30bb3a11aa6ca', 'request_id': 'req-5d93e475-75e1-4e6f-a8cd-d51e8cc9d0a9'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:47.922315 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-5d93e475-75e1-4e6f-a8cd-d51e8cc9d0a9 admin admin] The token cache is being invalidate because project 4eb1fc6b915b4e2399d30bb3a11aa6ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 19:18:47.922601 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-5d93e475-75e1-4e6f-a8cd-d51e8cc9d0a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:47.923242 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-5d93e475-75e1-4e6f-a8cd-d51e8cc9d0a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:47.924769 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 893/1781] 10.222.0.61 () {58 vars in 1233 bytes} [Mon Aug 22 19:18:47 2022] DELETE /identity/v3/projects/4eb1fc6b915b4e2399d30bb3a11aa6ca => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:18:47.928703 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c4e448-0593-4acc-9a42-ba726d38b1da service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:47.935403 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:18:47.939389 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c4e448-0593-4acc-9a42-ba726d38b1da service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:47.998038 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539056fa-8eb4-4de0-a381-c94f21893027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:47.998704 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539056fa-8eb4-4de0-a381-c94f21893027 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:47.999126 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539056fa-8eb4-4de0-a381-c94f21893027 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:47.999406 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539056fa-8eb4-4de0-a381-c94f21893027 admin admin] PATH_INFO: `/v3/users/b1dfd418a9cb4e35b5be038a8b0a1b57` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:48.000566 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539056fa-8eb4-4de0-a381-c94f21893027 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1dfd418a9cb4e35b5be038a8b0a1b57)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:48.002476 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539056fa-8eb4-4de0-a381-c94f21893027 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:48.048490 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-539056fa-8eb4-4de0-a381-c94f21893027 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1dfd418a9cb4e35b5be038a8b0a1b57'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:48.049122 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-539056fa-8eb4-4de0-a381-c94f21893027 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1dfd418a9cb4e35b5be038a8b0a1b57'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:48.055951 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-539056fa-8eb4-4de0-a381-c94f21893027 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1dfd418a9cb4e35b5be038a8b0a1b57'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:48.063403 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-539056fa-8eb4-4de0-a381-c94f21893027 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1dfd418a9cb4e35b5be038a8b0a1b57'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:48.075661 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-539056fa-8eb4-4de0-a381-c94f21893027 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1dfd418a9cb4e35b5be038a8b0a1b57'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:48.081375 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 889/1782] 10.222.0.61 () {58 vars in 1224 bytes} [Mon Aug 22 19:18:47 2022] DELETE /identity/v3/users/b1dfd418a9cb4e35b5be038a8b0a1b57 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 19:18:48.352026 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d93e475-75e1-4e6f-a8cd-d51e8cc9d0a9 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:48.353638 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-5d93e475-75e1-4e6f-a8cd-d51e8cc9d0a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:48.381763 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d93e475-75e1-4e6f-a8cd-d51e8cc9d0a9 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:48.435515 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:18:48.467571 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea32b36d-f4bd-4489-bdac-702368e41250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:48.468248 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea32b36d-f4bd-4489-bdac-702368e41250 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:48.468529 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea32b36d-f4bd-4489-bdac-702368e41250 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:48.468803 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea32b36d-f4bd-4489-bdac-702368e41250 admin admin] PATH_INFO: `/v3/projects/80cba7f2f9f448dc8e3f0a33d8e88a9f` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:48.469931 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea32b36d-f4bd-4489-bdac-702368e41250 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80cba7f2f9f448dc8e3f0a33d8e88a9f)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:48.471941 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea32b36d-f4bd-4489-bdac-702368e41250 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:48.535574 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-ea32b36d-f4bd-4489-bdac-702368e41250 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80cba7f2f9f448dc8e3f0a33d8e88a9f', 'request_id': 'req-ea32b36d-f4bd-4489-bdac-702368e41250'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:48.544036 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-ea32b36d-f4bd-4489-bdac-702368e41250 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80cba7f2f9f448dc8e3f0a33d8e88a9f', 'request_id': 'req-ea32b36d-f4bd-4489-bdac-702368e41250'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:48.550691 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-ea32b36d-f4bd-4489-bdac-702368e41250 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80cba7f2f9f448dc8e3f0a33d8e88a9f', 'request_id': 'req-ea32b36d-f4bd-4489-bdac-702368e41250'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:48.565092 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-ea32b36d-f4bd-4489-bdac-702368e41250 admin admin] The token cache is being invalidate because project 80cba7f2f9f448dc8e3f0a33d8e88a9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 19:18:48.565385 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-ea32b36d-f4bd-4489-bdac-702368e41250 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:48.566310 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-ea32b36d-f4bd-4489-bdac-702368e41250 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:48.571608 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 894/1783] 10.222.0.61 () {58 vars in 1233 bytes} [Mon Aug 22 19:18:48 2022] DELETE /identity/v3/projects/80cba7f2f9f448dc8e3f0a33d8e88a9f => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:18:48.574832 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4b0928-5602-4813-822f-e4a4b7483a7b None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:48.575443 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4b0928-5602-4813-822f-e4a4b7483a7b None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:48.575620 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4b0928-5602-4813-822f-e4a4b7483a7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:48.604081 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-bd4b0928-5602-4813-822f-e4a4b7483a7b None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:18:48.605404 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-bd4b0928-5602-4813-822f-e4a4b7483a7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:48.669083 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 890/1784] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:18:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:18:48.676694 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ac5834-2bf0-41be-8b43-7c8a301c515c None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:48.676980 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ac5834-2bf0-41be-8b43-7c8a301c515c None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:48.677252 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ac5834-2bf0-41be-8b43-7c8a301c515c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:48.697349 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-b7ac5834-2bf0-41be-8b43-7c8a301c515c None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:18:48.699054 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-b7ac5834-2bf0-41be-8b43-7c8a301c515c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:48.706943 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 895/1785] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:18:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:18:48.710955 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4b0928-5602-4813-822f-e4a4b7483a7b None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:48.720576 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4b0928-5602-4813-822f-e4a4b7483a7b None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:48.725155 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6ba59c-eb82-4ce1-b313-be0268003743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:48.725742 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6ba59c-eb82-4ce1-b313-be0268003743 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:48.726018 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6ba59c-eb82-4ce1-b313-be0268003743 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:48.726285 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6ba59c-eb82-4ce1-b313-be0268003743 admin admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:48.727191 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6ba59c-eb82-4ce1-b313-be0268003743 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:18:48.727486 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6ba59c-eb82-4ce1-b313-be0268003743 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:48.729090 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6ba59c-eb82-4ce1-b313-be0268003743 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:48.740406 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 891/1786] 10.222.0.61 () {58 vars in 1153 bytes} [Mon Aug 22 19:18:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:18:48.744480 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ac5834-2bf0-41be-8b43-7c8a301c515c None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:48.754193 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ac5834-2bf0-41be-8b43-7c8a301c515c None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:48.758783 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbacb8c-a0f8-477d-964d-c7fc5a7603f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:48.759370 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbacb8c-a0f8-477d-964d-c7fc5a7603f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:48.759738 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbacb8c-a0f8-477d-964d-c7fc5a7603f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:48.760490 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbacb8c-a0f8-477d-964d-c7fc5a7603f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:48.760587 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbacb8c-a0f8-477d-964d-c7fc5a7603f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:48.763094 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbacb8c-a0f8-477d-964d-c7fc5a7603f6 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:48.790291 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 896/1787] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 19:18:48 2022] POST /identity/v3/projects => generated 379 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 19:18:48.794451 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6ba59c-eb82-4ce1-b313-be0268003743 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:48.805530 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6ba59c-eb82-4ce1-b313-be0268003743 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:48.814032 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:18:48.866269 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be651d9-79ff-48f3-b92e-6b73167b7376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:48.866889 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be651d9-79ff-48f3-b92e-6b73167b7376 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:48.867168 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be651d9-79ff-48f3-b92e-6b73167b7376 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:48.867452 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be651d9-79ff-48f3-b92e-6b73167b7376 admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:48.868116 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be651d9-79ff-48f3-b92e-6b73167b7376 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:48.870014 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be651d9-79ff-48f3-b92e-6b73167b7376 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:48.895472 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 892/1788] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 19:18:48 2022] POST /identity/v3/users => generated 290 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:18:48.897199 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbacb8c-a0f8-477d-964d-c7fc5a7603f6 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:48.908289 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbacb8c-a0f8-477d-964d-c7fc5a7603f6 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:48.912992 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10220dda-7751-430d-9f0a-e37526f896f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:48.913677 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10220dda-7751-430d-9f0a-e37526f896f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:48.914020 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10220dda-7751-430d-9f0a-e37526f896f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:48.914356 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10220dda-7751-430d-9f0a-e37526f896f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:48.915060 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10220dda-7751-430d-9f0a-e37526f896f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:48.916747 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10220dda-7751-430d-9f0a-e37526f896f5 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:48.927460 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 897/1789] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:18:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:18:48.931599 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be651d9-79ff-48f3-b92e-6b73167b7376 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:48.942255 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be651d9-79ff-48f3-b92e-6b73167b7376 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:48.946886 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b145f8b-3e54-4113-bb14-fa75d8c7423d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:48.947553 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b145f8b-3e54-4113-bb14-fa75d8c7423d admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:48.947807 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b145f8b-3e54-4113-bb14-fa75d8c7423d admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:48.948083 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b145f8b-3e54-4113-bb14-fa75d8c7423d admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:48.948762 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b145f8b-3e54-4113-bb14-fa75d8c7423d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:48.950425 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b145f8b-3e54-4113-bb14-fa75d8c7423d admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:48.962295 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 893/1790] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:18:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:18:48.966393 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10220dda-7751-430d-9f0a-e37526f896f5 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:48.977872 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10220dda-7751-430d-9f0a-e37526f896f5 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:48.983922 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e315c18-d636-4d94-b314-3084e72c6b1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:48.984667 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e315c18-d636-4d94-b314-3084e72c6b1a admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:48.985025 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e315c18-d636-4d94-b314-3084e72c6b1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:48.985363 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e315c18-d636-4d94-b314-3084e72c6b1a admin admin] PATH_INFO: `/v3/projects/51c41d2a2d71449d8e1c1c28f2726045/users/762bc184687c4bf387387cea07414618/roles/74a40bc819cb42eea7e966a4b2a016f1` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:49.004400 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e315c18-d636-4d94-b314-3084e72c6b1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=51c41d2a2d71449d8e1c1c28f2726045, user_id=762bc184687c4bf387387cea07414618, role_id=74a40bc819cb42eea7e966a4b2a016f1)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:49.007083 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e315c18-d636-4d94-b314-3084e72c6b1a admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:49.021946 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 898/1791] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:18:48 2022] PUT /identity/v3/projects/51c41d2a2d71449d8e1c1c28f2726045/users/762bc184687c4bf387387cea07414618/roles/74a40bc819cb42eea7e966a4b2a016f1 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:18:49.032139 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415b5189-c315-46d4-be90-abd51d7c3f36 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:49.032426 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415b5189-c315-46d4-be90-abd51d7c3f36 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:49.032699 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415b5189-c315-46d4-be90-abd51d7c3f36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:49.060616 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-415b5189-c315-46d4-be90-abd51d7c3f36 None None] MFA Rules not processed for user `762bc184687c4bf387387cea07414618`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:18:49.061831 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-415b5189-c315-46d4-be90-abd51d7c3f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:49.124973 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 894/1792] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:18:49 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:18:49.129969 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e315c18-d636-4d94-b314-3084e72c6b1a admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:49.193599 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e315c18-d636-4d94-b314-3084e72c6b1a admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:49.198257 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112d6288-8b0d-4756-839e-6b9236d8bed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:49.198803 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112d6288-8b0d-4756-839e-6b9236d8bed5 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:49.199079 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112d6288-8b0d-4756-839e-6b9236d8bed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:49.199347 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112d6288-8b0d-4756-839e-6b9236d8bed5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:49.200015 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112d6288-8b0d-4756-839e-6b9236d8bed5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:49.201918 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112d6288-8b0d-4756-839e-6b9236d8bed5 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:49.206316 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:18:49.231862 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 899/1793] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 19:18:49 2022] POST /identity/v3/projects => generated 381 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:18:49.237516 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415b5189-c315-46d4-be90-abd51d7c3f36 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:49.254119 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415b5189-c315-46d4-be90-abd51d7c3f36 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:49.310007 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493c7bce-de74-453d-bb14-7b60caba1863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:49.310610 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493c7bce-de74-453d-bb14-7b60caba1863 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:49.310889 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493c7bce-de74-453d-bb14-7b60caba1863 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:49.311157 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493c7bce-de74-453d-bb14-7b60caba1863 admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:49.311850 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493c7bce-de74-453d-bb14-7b60caba1863 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:49.313738 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493c7bce-de74-453d-bb14-7b60caba1863 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:49.333832 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 895/1794] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 19:18:49 2022] POST /identity/v3/users => generated 297 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:18:49.337865 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112d6288-8b0d-4756-839e-6b9236d8bed5 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:49.348629 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112d6288-8b0d-4756-839e-6b9236d8bed5 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:49.353381 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46a962b-4c8a-4165-a310-3715c7305e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:49.353974 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46a962b-4c8a-4165-a310-3715c7305e4a admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:49.354252 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46a962b-4c8a-4165-a310-3715c7305e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:49.354524 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46a962b-4c8a-4165-a310-3715c7305e4a admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:49.355744 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46a962b-4c8a-4165-a310-3715c7305e4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:49.357337 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46a962b-4c8a-4165-a310-3715c7305e4a admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:49.367154 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 900/1795] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:18:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:18:49.371124 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493c7bce-de74-453d-bb14-7b60caba1863 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:49.381099 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493c7bce-de74-453d-bb14-7b60caba1863 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:49.385613 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f455a424-ff0b-41b3-bfd3-c42deecb6605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:49.386252 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f455a424-ff0b-41b3-bfd3-c42deecb6605 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:49.386533 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f455a424-ff0b-41b3-bfd3-c42deecb6605 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:49.386811 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f455a424-ff0b-41b3-bfd3-c42deecb6605 admin admin] PATH_INFO: `/v3/projects/07bdddd9885748159a3e5180ce0389e0/users/6ae70d5a82a842bf872e40bc4ea77863/roles/163c30c3b3c246beb849ce176f02d9da` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:49.389813 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:18:49.402931 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f455a424-ff0b-41b3-bfd3-c42deecb6605 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07bdddd9885748159a3e5180ce0389e0, user_id=6ae70d5a82a842bf872e40bc4ea77863, role_id=163c30c3b3c246beb849ce176f02d9da)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:49.405817 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f455a424-ff0b-41b3-bfd3-c42deecb6605 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:49.418439 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 896/1796] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:18:49 2022] PUT /identity/v3/projects/07bdddd9885748159a3e5180ce0389e0/users/6ae70d5a82a842bf872e40bc4ea77863/roles/163c30c3b3c246beb849ce176f02d9da => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:18:49.422285 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46a962b-4c8a-4165-a310-3715c7305e4a admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:49.432165 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46a962b-4c8a-4165-a310-3715c7305e4a admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:49.490598 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6198d0c-4342-4ca8-9f01-f3967eb4d618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:49.491212 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6198d0c-4342-4ca8-9f01-f3967eb4d618 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:49.491488 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6198d0c-4342-4ca8-9f01-f3967eb4d618 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:49.491789 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6198d0c-4342-4ca8-9f01-f3967eb4d618 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:49.492430 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6198d0c-4342-4ca8-9f01-f3967eb4d618 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:49.494022 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6198d0c-4342-4ca8-9f01-f3967eb4d618 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:49.504499 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 901/1797] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:18:49 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:18:49.508879 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f455a424-ff0b-41b3-bfd3-c42deecb6605 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:49.519299 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f455a424-ff0b-41b3-bfd3-c42deecb6605 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:49.524035 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8557780-4866-4977-b6f4-26f68792f89c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:49.524796 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8557780-4866-4977-b6f4-26f68792f89c admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:49.525136 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8557780-4866-4977-b6f4-26f68792f89c admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:49.525569 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8557780-4866-4977-b6f4-26f68792f89c admin admin] PATH_INFO: `/v3/projects/07bdddd9885748159a3e5180ce0389e0/users/6ae70d5a82a842bf872e40bc4ea77863/roles/163c30c3b3c246beb849ce176f02d9da` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:49.528109 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8557780-4866-4977-b6f4-26f68792f89c admin admin] RBAC: Authorizing `identity:create_grant(project_id=07bdddd9885748159a3e5180ce0389e0, user_id=6ae70d5a82a842bf872e40bc4ea77863, role_id=163c30c3b3c246beb849ce176f02d9da)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:49.532414 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8557780-4866-4977-b6f4-26f68792f89c admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:49.543147 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 897/1798] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:18:49 2022] PUT /identity/v3/projects/07bdddd9885748159a3e5180ce0389e0/users/6ae70d5a82a842bf872e40bc4ea77863/roles/163c30c3b3c246beb849ce176f02d9da => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:18:49.554465 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3098160e-40b3-432f-80e2-b0e47a7111a2 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:49.554647 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3098160e-40b3-432f-80e2-b0e47a7111a2 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:49.555096 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3098160e-40b3-432f-80e2-b0e47a7111a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:49.594974 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-3098160e-40b3-432f-80e2-b0e47a7111a2 None None] MFA Rules not processed for user `6ae70d5a82a842bf872e40bc4ea77863`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:18:49.596239 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-3098160e-40b3-432f-80e2-b0e47a7111a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:49.679172 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 902/1799] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:18:49 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:18:49.691444 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8557780-4866-4977-b6f4-26f68792f89c admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:49.694253 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-d8557780-4866-4977-b6f4-26f68792f89c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:49.759818 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8557780-4866-4977-b6f4-26f68792f89c admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:49.763610 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ee40aa-2e85-4689-bb0d-add37383b8f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e79498873d4ed29a0e19bbea9b9214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:49.764247 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ee40aa-2e85-4689-bb0d-add37383b8f3 service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:49.764528 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ee40aa-2e85-4689-bb0d-add37383b8f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:49.764800 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ee40aa-2e85-4689-bb0d-add37383b8f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:49.774740 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ee40aa-2e85-4689-bb0d-add37383b8f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:49.776201 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ee40aa-2e85-4689-bb0d-add37383b8f3 service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:49.786837 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 898/1800] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:18:49 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:18:50.134187 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3098160e-40b3-432f-80e2-b0e47a7111a2 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:50.144627 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3098160e-40b3-432f-80e2-b0e47a7111a2 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:50.150710 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a3d99e-7df2-4126-9be6-9b48ec07e9f0 tempest-VolumeQuotaClassesTest-2065899745 tempest-VolumeQuotaClassesTest-2065899745-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ae70d5a82a842bf872e40bc4ea77863', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bdddd9885748159a3e5180ce0389e0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:50.151340 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a3d99e-7df2-4126-9be6-9b48ec07e9f0 tempest-VolumeQuotaClassesTest-2065899745 tempest-VolumeQuotaClassesTest-2065899745-project-admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:50.151670 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a3d99e-7df2-4126-9be6-9b48ec07e9f0 tempest-VolumeQuotaClassesTest-2065899745 tempest-VolumeQuotaClassesTest-2065899745-project-admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:50.151950 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a3d99e-7df2-4126-9be6-9b48ec07e9f0 tempest-VolumeQuotaClassesTest-2065899745 tempest-VolumeQuotaClassesTest-2065899745-project-admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:50.152610 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a3d99e-7df2-4126-9be6-9b48ec07e9f0 tempest-VolumeQuotaClassesTest-2065899745 tempest-VolumeQuotaClassesTest-2065899745-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:18:50.152911 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a3d99e-7df2-4126-9be6-9b48ec07e9f0 tempest-VolumeQuotaClassesTest-2065899745 tempest-VolumeQuotaClassesTest-2065899745-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:50.154507 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a3d99e-7df2-4126-9be6-9b48ec07e9f0 tempest-VolumeQuotaClassesTest-2065899745 tempest-VolumeQuotaClassesTest-2065899745-project-admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:50.167269 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 903/1801] 10.222.0.61 () {58 vars in 1153 bytes} [Mon Aug 22 19:18:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:18:50.171670 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ee40aa-2e85-4689-bb0d-add37383b8f3 service cinder] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:50.186183 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ee40aa-2e85-4689-bb0d-add37383b8f3 service cinder] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:50.191697 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e285c1-6cf6-41c6-b12b-b22ea7e5bbe2 tempest-VolumeQuotaClassesTest-2065899745 tempest-VolumeQuotaClassesTest-2065899745-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ae70d5a82a842bf872e40bc4ea77863', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bdddd9885748159a3e5180ce0389e0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:50.192078 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e285c1-6cf6-41c6-b12b-b22ea7e5bbe2 tempest-VolumeQuotaClassesTest-2065899745 tempest-VolumeQuotaClassesTest-2065899745-project-admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:50.192363 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e285c1-6cf6-41c6-b12b-b22ea7e5bbe2 tempest-VolumeQuotaClassesTest-2065899745 tempest-VolumeQuotaClassesTest-2065899745-project-admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:50.192647 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e285c1-6cf6-41c6-b12b-b22ea7e5bbe2 tempest-VolumeQuotaClassesTest-2065899745 tempest-VolumeQuotaClassesTest-2065899745-project-admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:50.193283 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e285c1-6cf6-41c6-b12b-b22ea7e5bbe2 tempest-VolumeQuotaClassesTest-2065899745 tempest-VolumeQuotaClassesTest-2065899745-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:50.195479 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e285c1-6cf6-41c6-b12b-b22ea7e5bbe2 tempest-VolumeQuotaClassesTest-2065899745 tempest-VolumeQuotaClassesTest-2065899745-project-admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:50.227273 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 899/1802] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 19:18:50 2022] POST /identity/v3/projects => generated 353 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:18:50.231420 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a3d99e-7df2-4126-9be6-9b48ec07e9f0 tempest-VolumeQuotaClassesTest-2065899745 tempest-VolumeQuotaClassesTest-2065899745-project-admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:50.248199 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a3d99e-7df2-4126-9be6-9b48ec07e9f0 tempest-VolumeQuotaClassesTest-2065899745 tempest-VolumeQuotaClassesTest-2065899745-project-admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:50.297479 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:18:50.317922 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0de49be-3680-4017-bfba-b4958b6e67cf tempest-VolumeQuotaClassesTest-2065899745 tempest-VolumeQuotaClassesTest-2065899745-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ae70d5a82a842bf872e40bc4ea77863', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bdddd9885748159a3e5180ce0389e0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:50.318562 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0de49be-3680-4017-bfba-b4958b6e67cf tempest-VolumeQuotaClassesTest-2065899745 tempest-VolumeQuotaClassesTest-2065899745-project-admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:50.319114 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0de49be-3680-4017-bfba-b4958b6e67cf tempest-VolumeQuotaClassesTest-2065899745 tempest-VolumeQuotaClassesTest-2065899745-project-admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:50.319393 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0de49be-3680-4017-bfba-b4958b6e67cf tempest-VolumeQuotaClassesTest-2065899745 tempest-VolumeQuotaClassesTest-2065899745-project-admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:50.320075 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0de49be-3680-4017-bfba-b4958b6e67cf tempest-VolumeQuotaClassesTest-2065899745 tempest-VolumeQuotaClassesTest-2065899745-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:18:50.320381 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0de49be-3680-4017-bfba-b4958b6e67cf tempest-VolumeQuotaClassesTest-2065899745 tempest-VolumeQuotaClassesTest-2065899745-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:50.322020 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0de49be-3680-4017-bfba-b4958b6e67cf tempest-VolumeQuotaClassesTest-2065899745 tempest-VolumeQuotaClassesTest-2065899745-project-admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:50.334262 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 904/1803] 10.222.0.61 () {58 vars in 1153 bytes} [Mon Aug 22 19:18:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:18:50.375354 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e285c1-6cf6-41c6-b12b-b22ea7e5bbe2 tempest-VolumeQuotaClassesTest-2065899745 tempest-VolumeQuotaClassesTest-2065899745-project-admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:50.385580 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e285c1-6cf6-41c6-b12b-b22ea7e5bbe2 tempest-VolumeQuotaClassesTest-2065899745 tempest-VolumeQuotaClassesTest-2065899745-project-admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:50.390236 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cb2cc6-fab2-4cb6-8c18-bd0af0fe18e2 tempest-VolumeQuotaClassesTest-2065899745 tempest-VolumeQuotaClassesTest-2065899745-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ae70d5a82a842bf872e40bc4ea77863', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07bdddd9885748159a3e5180ce0389e0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:50.390865 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cb2cc6-fab2-4cb6-8c18-bd0af0fe18e2 tempest-VolumeQuotaClassesTest-2065899745 tempest-VolumeQuotaClassesTest-2065899745-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:50.391147 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cb2cc6-fab2-4cb6-8c18-bd0af0fe18e2 tempest-VolumeQuotaClassesTest-2065899745 tempest-VolumeQuotaClassesTest-2065899745-project-admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:50.391419 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cb2cc6-fab2-4cb6-8c18-bd0af0fe18e2 tempest-VolumeQuotaClassesTest-2065899745 tempest-VolumeQuotaClassesTest-2065899745-project-admin] PATH_INFO: `/v3/projects/38be3307700b4e498bba2ce6b8823e3a` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:50.392592 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cb2cc6-fab2-4cb6-8c18-bd0af0fe18e2 tempest-VolumeQuotaClassesTest-2065899745 tempest-VolumeQuotaClassesTest-2065899745-project-admin] RBAC: Authorizing `identity:delete_project(project_id=38be3307700b4e498bba2ce6b8823e3a)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:50.394479 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cb2cc6-fab2-4cb6-8c18-bd0af0fe18e2 tempest-VolumeQuotaClassesTest-2065899745 tempest-VolumeQuotaClassesTest-2065899745-project-admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:50.443098 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a7cb2cc6-fab2-4cb6-8c18-bd0af0fe18e2 tempest-VolumeQuotaClassesTest-2065899745 tempest-VolumeQuotaClassesTest-2065899745-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38be3307700b4e498bba2ce6b8823e3a', 'request_id': 'req-a7cb2cc6-fab2-4cb6-8c18-bd0af0fe18e2'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:50.451445 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a7cb2cc6-fab2-4cb6-8c18-bd0af0fe18e2 tempest-VolumeQuotaClassesTest-2065899745 tempest-VolumeQuotaClassesTest-2065899745-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38be3307700b4e498bba2ce6b8823e3a', 'request_id': 'req-a7cb2cc6-fab2-4cb6-8c18-bd0af0fe18e2'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:50.461936 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a7cb2cc6-fab2-4cb6-8c18-bd0af0fe18e2 tempest-VolumeQuotaClassesTest-2065899745 tempest-VolumeQuotaClassesTest-2065899745-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38be3307700b4e498bba2ce6b8823e3a', 'request_id': 'req-a7cb2cc6-fab2-4cb6-8c18-bd0af0fe18e2'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:50.477130 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a7cb2cc6-fab2-4cb6-8c18-bd0af0fe18e2 tempest-VolumeQuotaClassesTest-2065899745 tempest-VolumeQuotaClassesTest-2065899745-project-admin] The token cache is being invalidate because project 38be3307700b4e498bba2ce6b8823e3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 19:18:50.477533 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a7cb2cc6-fab2-4cb6-8c18-bd0af0fe18e2 tempest-VolumeQuotaClassesTest-2065899745 tempest-VolumeQuotaClassesTest-2065899745-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:50.478103 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a7cb2cc6-fab2-4cb6-8c18-bd0af0fe18e2 tempest-VolumeQuotaClassesTest-2065899745 tempest-VolumeQuotaClassesTest-2065899745-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:50.483905 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 900/1804] 10.222.0.61 () {58 vars in 1233 bytes} [Mon Aug 22 19:18:50 2022] DELETE /identity/v3/projects/38be3307700b4e498bba2ce6b8823e3a => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:18:50.785116 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0de49be-3680-4017-bfba-b4958b6e67cf tempest-VolumeQuotaClassesTest-2065899745 tempest-VolumeQuotaClassesTest-2065899745-project-admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:50.787047 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-b0de49be-3680-4017-bfba-b4958b6e67cf tempest-VolumeQuotaClassesTest-2065899745 tempest-VolumeQuotaClassesTest-2065899745-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:50.869672 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0de49be-3680-4017-bfba-b4958b6e67cf tempest-VolumeQuotaClassesTest-2065899745 tempest-VolumeQuotaClassesTest-2065899745-project-admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:50.948589 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f2eb79-1f57-44d9-968b-80335c6cf161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:50.950780 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f2eb79-1f57-44d9-968b-80335c6cf161 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:50.950780 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f2eb79-1f57-44d9-968b-80335c6cf161 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:50.950780 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f2eb79-1f57-44d9-968b-80335c6cf161 admin admin] PATH_INFO: `/v3/users/762bc184687c4bf387387cea07414618` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:50.951058 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f2eb79-1f57-44d9-968b-80335c6cf161 admin admin] RBAC: Authorizing `identity:delete_user(user_id=762bc184687c4bf387387cea07414618)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:50.952985 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f2eb79-1f57-44d9-968b-80335c6cf161 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:51.003496 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-b3f2eb79-1f57-44d9-968b-80335c6cf161 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '762bc184687c4bf387387cea07414618'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:51.004315 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-b3f2eb79-1f57-44d9-968b-80335c6cf161 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '762bc184687c4bf387387cea07414618'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:51.012368 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-b3f2eb79-1f57-44d9-968b-80335c6cf161 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '762bc184687c4bf387387cea07414618'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:51.019811 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-b3f2eb79-1f57-44d9-968b-80335c6cf161 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '762bc184687c4bf387387cea07414618'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:51.033082 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-b3f2eb79-1f57-44d9-968b-80335c6cf161 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '762bc184687c4bf387387cea07414618'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:51.039101 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 905/1805] 10.222.0.61 () {58 vars in 1224 bytes} [Mon Aug 22 19:18:50 2022] DELETE /identity/v3/users/762bc184687c4bf387387cea07414618 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:18:51.044064 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23b4ed6-9a4a-4076-9ef0-ce39cbef917c None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:51.044348 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23b4ed6-9a4a-4076-9ef0-ce39cbef917c None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:51.044623 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23b4ed6-9a4a-4076-9ef0-ce39cbef917c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:51.063181 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-e23b4ed6-9a4a-4076-9ef0-ce39cbef917c None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:18:51.064434 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-e23b4ed6-9a4a-4076-9ef0-ce39cbef917c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:51.094290 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:18:51.125714 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 901/1806] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:18:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:18:51.135257 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f2eb79-1f57-44d9-968b-80335c6cf161 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:51.137237 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-b3f2eb79-1f57-44d9-968b-80335c6cf161 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:51.143122 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:18:51.196902 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f2eb79-1f57-44d9-968b-80335c6cf161 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:51.201027 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd654b18-4a34-4590-be48-0c633ba76784 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:51.201634 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd654b18-4a34-4590-be48-0c633ba76784 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:51.201907 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd654b18-4a34-4590-be48-0c633ba76784 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:51.202176 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd654b18-4a34-4590-be48-0c633ba76784 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:51.210906 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd654b18-4a34-4590-be48-0c633ba76784 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:51.212367 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd654b18-4a34-4590-be48-0c633ba76784 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:51.222136 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 906/1807] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:18:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:18:51.523074 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23b4ed6-9a4a-4076-9ef0-ce39cbef917c None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:51.524796 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-e23b4ed6-9a4a-4076-9ef0-ce39cbef917c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:51.538217 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23b4ed6-9a4a-4076-9ef0-ce39cbef917c None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:51.542908 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db70b9f1-6615-496f-83af-84ac5219149f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:51.543740 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db70b9f1-6615-496f-83af-84ac5219149f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:51.544096 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db70b9f1-6615-496f-83af-84ac5219149f admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:51.544443 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db70b9f1-6615-496f-83af-84ac5219149f admin admin] PATH_INFO: `/v3/projects/51c41d2a2d71449d8e1c1c28f2726045` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:51.557572 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db70b9f1-6615-496f-83af-84ac5219149f admin admin] RBAC: Authorizing `identity:delete_project(project_id=51c41d2a2d71449d8e1c1c28f2726045)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:51.560141 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db70b9f1-6615-496f-83af-84ac5219149f admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:51.613421 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-db70b9f1-6615-496f-83af-84ac5219149f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51c41d2a2d71449d8e1c1c28f2726045', 'request_id': 'req-db70b9f1-6615-496f-83af-84ac5219149f'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:51.626536 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-db70b9f1-6615-496f-83af-84ac5219149f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51c41d2a2d71449d8e1c1c28f2726045', 'request_id': 'req-db70b9f1-6615-496f-83af-84ac5219149f'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:51.637276 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-db70b9f1-6615-496f-83af-84ac5219149f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51c41d2a2d71449d8e1c1c28f2726045', 'request_id': 'req-db70b9f1-6615-496f-83af-84ac5219149f'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:51.648051 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-db70b9f1-6615-496f-83af-84ac5219149f admin admin] The token cache is being invalidate because project 51c41d2a2d71449d8e1c1c28f2726045 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 19:18:51.648330 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-db70b9f1-6615-496f-83af-84ac5219149f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:51.649117 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-db70b9f1-6615-496f-83af-84ac5219149f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:51.650643 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 902/1808] 10.222.0.61 () {58 vars in 1233 bytes} [Mon Aug 22 19:18:51 2022] DELETE /identity/v3/projects/51c41d2a2d71449d8e1c1c28f2726045 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:18:51.654620 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd654b18-4a34-4590-be48-0c633ba76784 service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:51.664537 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd654b18-4a34-4590-be48-0c633ba76784 service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:51.666904 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db70b9f1-6615-496f-83af-84ac5219149f admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:51.668490 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-db70b9f1-6615-496f-83af-84ac5219149f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:51.723467 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9384fa-b584-456d-a8b5-ee89d9a95d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:51.724163 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9384fa-b584-456d-a8b5-ee89d9a95d70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:51.724441 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9384fa-b584-456d-a8b5-ee89d9a95d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:51.724713 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9384fa-b584-456d-a8b5-ee89d9a95d70 admin admin] PATH_INFO: `/v3/users/6ae70d5a82a842bf872e40bc4ea77863` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:51.725980 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9384fa-b584-456d-a8b5-ee89d9a95d70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ae70d5a82a842bf872e40bc4ea77863)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:51.728295 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9384fa-b584-456d-a8b5-ee89d9a95d70 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:51.732285 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db70b9f1-6615-496f-83af-84ac5219149f admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:51.736825 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f2b03f-3cf2-4240-bcc8-9151a5e4995f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:51.737449 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f2b03f-3cf2-4240-bcc8-9151a5e4995f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:51.737724 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f2b03f-3cf2-4240-bcc8-9151a5e4995f admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:51.737992 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f2b03f-3cf2-4240-bcc8-9151a5e4995f admin admin] PATH_INFO: `/v3/users/6331c5ab05c14a00ac4053b6bab86650` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:51.739517 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f2b03f-3cf2-4240-bcc8-9151a5e4995f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6331c5ab05c14a00ac4053b6bab86650)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:51.741445 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f2b03f-3cf2-4240-bcc8-9151a5e4995f admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:51.782003 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-ec9384fa-b584-456d-a8b5-ee89d9a95d70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ae70d5a82a842bf872e40bc4ea77863'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:51.782724 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-ec9384fa-b584-456d-a8b5-ee89d9a95d70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ae70d5a82a842bf872e40bc4ea77863'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:51.791328 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-ec9384fa-b584-456d-a8b5-ee89d9a95d70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ae70d5a82a842bf872e40bc4ea77863'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:51.796032 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-91f2b03f-3cf2-4240-bcc8-9151a5e4995f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6331c5ab05c14a00ac4053b6bab86650'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:51.797098 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-91f2b03f-3cf2-4240-bcc8-9151a5e4995f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6331c5ab05c14a00ac4053b6bab86650'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:51.803728 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-ec9384fa-b584-456d-a8b5-ee89d9a95d70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ae70d5a82a842bf872e40bc4ea77863'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:51.822049 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-ec9384fa-b584-456d-a8b5-ee89d9a95d70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ae70d5a82a842bf872e40bc4ea77863'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:51.822200 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-91f2b03f-3cf2-4240-bcc8-9151a5e4995f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6331c5ab05c14a00ac4053b6bab86650'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:51.827345 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:18:51.833782 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-91f2b03f-3cf2-4240-bcc8-9151a5e4995f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6331c5ab05c14a00ac4053b6bab86650'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:51.838553 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 907/1809] 10.222.0.61 () {58 vars in 1224 bytes} [Mon Aug 22 19:18:51 2022] DELETE /identity/v3/users/6ae70d5a82a842bf872e40bc4ea77863 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:18:51.853988 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-91f2b03f-3cf2-4240-bcc8-9151a5e4995f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6331c5ab05c14a00ac4053b6bab86650'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:51.867129 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 903/1810] 10.222.0.61 () {58 vars in 1224 bytes} [Mon Aug 22 19:18:51 2022] DELETE /identity/v3/users/6331c5ab05c14a00ac4053b6bab86650 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 19:18:51.869218 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8155ad-a7e1-4450-8646-c2153f54d47a None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:51.869514 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8155ad-a7e1-4450-8646-c2153f54d47a None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:51.869817 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8155ad-a7e1-4450-8646-c2153f54d47a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:51.894558 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-3b8155ad-a7e1-4450-8646-c2153f54d47a None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:18:51.899933 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-3b8155ad-a7e1-4450-8646-c2153f54d47a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:51.938901 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:18:52.003437 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 908/1811] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:18:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:18:52.018990 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f2b03f-3cf2-4240-bcc8-9151a5e4995f admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:52.020920 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-91f2b03f-3cf2-4240-bcc8-9151a5e4995f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:52.106162 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f2b03f-3cf2-4240-bcc8-9151a5e4995f admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:52.116253 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b691b4d-2d36-446d-a461-015072ddefd1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:52.117587 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b691b4d-2d36-446d-a461-015072ddefd1 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:52.117896 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b691b4d-2d36-446d-a461-015072ddefd1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:52.118309 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b691b4d-2d36-446d-a461-015072ddefd1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:52.132615 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b691b4d-2d36-446d-a461-015072ddefd1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:52.134088 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b691b4d-2d36-446d-a461-015072ddefd1 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:52.152019 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 904/1812] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:18:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:18:52.323346 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8155ad-a7e1-4450-8646-c2153f54d47a None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:52.326077 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-3b8155ad-a7e1-4450-8646-c2153f54d47a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:52.341972 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8155ad-a7e1-4450-8646-c2153f54d47a None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:52.347814 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95061dd6-f652-4f32-93c7-d7960065a537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:52.348369 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95061dd6-f652-4f32-93c7-d7960065a537 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:52.348649 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95061dd6-f652-4f32-93c7-d7960065a537 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:52.348922 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95061dd6-f652-4f32-93c7-d7960065a537 admin admin] PATH_INFO: `/v3/projects/07bdddd9885748159a3e5180ce0389e0` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:52.350091 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95061dd6-f652-4f32-93c7-d7960065a537 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07bdddd9885748159a3e5180ce0389e0)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:52.352316 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95061dd6-f652-4f32-93c7-d7960065a537 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:52.419221 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-95061dd6-f652-4f32-93c7-d7960065a537 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07bdddd9885748159a3e5180ce0389e0', 'request_id': 'req-95061dd6-f652-4f32-93c7-d7960065a537'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:52.428815 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-95061dd6-f652-4f32-93c7-d7960065a537 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07bdddd9885748159a3e5180ce0389e0', 'request_id': 'req-95061dd6-f652-4f32-93c7-d7960065a537'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:52.436956 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-95061dd6-f652-4f32-93c7-d7960065a537 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07bdddd9885748159a3e5180ce0389e0', 'request_id': 'req-95061dd6-f652-4f32-93c7-d7960065a537'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:52.450626 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-95061dd6-f652-4f32-93c7-d7960065a537 admin admin] The token cache is being invalidate because project 07bdddd9885748159a3e5180ce0389e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 19:18:52.450881 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-95061dd6-f652-4f32-93c7-d7960065a537 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:52.451540 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-95061dd6-f652-4f32-93c7-d7960065a537 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:52.456535 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 909/1813] 10.222.0.61 () {58 vars in 1233 bytes} [Mon Aug 22 19:18:52 2022] DELETE /identity/v3/projects/07bdddd9885748159a3e5180ce0389e0 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:18:52.458176 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d09035b-bd8b-420d-bfab-3a4fac8df9e1 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:52.458477 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d09035b-bd8b-420d-bfab-3a4fac8df9e1 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:52.458741 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d09035b-bd8b-420d-bfab-3a4fac8df9e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:52.479506 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-1d09035b-bd8b-420d-bfab-3a4fac8df9e1 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:18:52.481328 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-1d09035b-bd8b-420d-bfab-3a4fac8df9e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:52.546566 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 905/1814] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:18:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:18:52.554936 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8dd05c-6fa1-4e7e-aa2b-51c72ef33ae0 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:52.555399 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8dd05c-6fa1-4e7e-aa2b-51c72ef33ae0 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:52.555699 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8dd05c-6fa1-4e7e-aa2b-51c72ef33ae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:52.564558 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d09035b-bd8b-420d-bfab-3a4fac8df9e1 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:52.566085 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-1d09035b-bd8b-420d-bfab-3a4fac8df9e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:52.576886 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-9b8dd05c-6fa1-4e7e-aa2b-51c72ef33ae0 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:18:52.578104 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-9b8dd05c-6fa1-4e7e-aa2b-51c72ef33ae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:52.581303 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d09035b-bd8b-420d-bfab-3a4fac8df9e1 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:52.586926 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0153a067-b1ea-42ee-9d3b-bb3e8e4c6f19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:52.587553 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 910/1815] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:18:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 19:18:52.591608 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0153a067-b1ea-42ee-9d3b-bb3e8e4c6f19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:52.591925 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0153a067-b1ea-42ee-9d3b-bb3e8e4c6f19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:52.592211 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0153a067-b1ea-42ee-9d3b-bb3e8e4c6f19 admin admin] PATH_INFO: `/v3/projects/53e36b333dc84f988f53422c76ba2946` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:52.592771 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8dd05c-6fa1-4e7e-aa2b-51c72ef33ae0 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:52.593467 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0153a067-b1ea-42ee-9d3b-bb3e8e4c6f19 admin admin] RBAC: Authorizing `identity:delete_project(project_id=53e36b333dc84f988f53422c76ba2946)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:52.595633 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0153a067-b1ea-42ee-9d3b-bb3e8e4c6f19 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:52.605436 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8dd05c-6fa1-4e7e-aa2b-51c72ef33ae0 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:52.614131 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c99d988-772b-45ef-a502-1025b4b2a62f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:52.615035 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c99d988-772b-45ef-a502-1025b4b2a62f admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:52.615394 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c99d988-772b-45ef-a502-1025b4b2a62f admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:52.615815 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c99d988-772b-45ef-a502-1025b4b2a62f admin admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:52.616892 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c99d988-772b-45ef-a502-1025b4b2a62f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:18:52.617261 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c99d988-772b-45ef-a502-1025b4b2a62f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:52.620107 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c99d988-772b-45ef-a502-1025b4b2a62f admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:52.635417 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 911/1816] 10.222.0.61 () {58 vars in 1153 bytes} [Mon Aug 22 19:18:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 19:18:52.640643 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c99d988-772b-45ef-a502-1025b4b2a62f admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:52.651090 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c99d988-772b-45ef-a502-1025b4b2a62f admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:52.660297 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-0153a067-b1ea-42ee-9d3b-bb3e8e4c6f19 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53e36b333dc84f988f53422c76ba2946', 'request_id': 'req-0153a067-b1ea-42ee-9d3b-bb3e8e4c6f19'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:52.669853 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-0153a067-b1ea-42ee-9d3b-bb3e8e4c6f19 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53e36b333dc84f988f53422c76ba2946', 'request_id': 'req-0153a067-b1ea-42ee-9d3b-bb3e8e4c6f19'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:52.676591 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-0153a067-b1ea-42ee-9d3b-bb3e8e4c6f19 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53e36b333dc84f988f53422c76ba2946', 'request_id': 'req-0153a067-b1ea-42ee-9d3b-bb3e8e4c6f19'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:52.687385 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-0153a067-b1ea-42ee-9d3b-bb3e8e4c6f19 admin admin] The token cache is being invalidate because project 53e36b333dc84f988f53422c76ba2946 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 19:18:52.687700 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-0153a067-b1ea-42ee-9d3b-bb3e8e4c6f19 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:52.688364 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-0153a067-b1ea-42ee-9d3b-bb3e8e4c6f19 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:52.689874 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 906/1817] 10.222.0.61 () {58 vars in 1233 bytes} [Mon Aug 22 19:18:52 2022] DELETE /identity/v3/projects/53e36b333dc84f988f53422c76ba2946 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 19:18:52.693640 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0153a067-b1ea-42ee-9d3b-bb3e8e4c6f19 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:52.694971 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-0153a067-b1ea-42ee-9d3b-bb3e8e4c6f19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:52.702081 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:18:52.711046 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c0626d-8674-4b9b-881d-8f3f2d22ac0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:52.711718 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c0626d-8674-4b9b-881d-8f3f2d22ac0d admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:52.711999 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c0626d-8674-4b9b-881d-8f3f2d22ac0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:52.712259 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c0626d-8674-4b9b-881d-8f3f2d22ac0d admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:52.712881 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c0626d-8674-4b9b-881d-8f3f2d22ac0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:52.715414 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c0626d-8674-4b9b-881d-8f3f2d22ac0d admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:52.747529 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 912/1818] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 19:18:52 2022] POST /identity/v3/projects => generated 399 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:18:52.748601 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c0626d-8674-4b9b-881d-8f3f2d22ac0d admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:52.750410 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-64c0626d-8674-4b9b-881d-8f3f2d22ac0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:52.763971 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:18:52.764915 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0153a067-b1ea-42ee-9d3b-bb3e8e4c6f19 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:52.816411 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c0626d-8674-4b9b-881d-8f3f2d22ac0d admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:52.821315 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519df575-91b1-4930-86c5-d39a5040eea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:52.821466 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1410d480-c321-4a6b-84b4-66da593578ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:52.821930 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519df575-91b1-4930-86c5-d39a5040eea9 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:52.822039 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1410d480-c321-4a6b-84b4-66da593578ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:52.822211 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519df575-91b1-4930-86c5-d39a5040eea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:52.822337 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1410d480-c321-4a6b-84b4-66da593578ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:52.822480 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519df575-91b1-4930-86c5-d39a5040eea9 admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:52.822602 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1410d480-c321-4a6b-84b4-66da593578ee admin admin] PATH_INFO: `/v3/users/b21ad8c810474c67a82f6a285ba1664e` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:52.823100 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519df575-91b1-4930-86c5-d39a5040eea9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:52.824205 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1410d480-c321-4a6b-84b4-66da593578ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=b21ad8c810474c67a82f6a285ba1664e)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:52.825568 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519df575-91b1-4930-86c5-d39a5040eea9 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:52.826085 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1410d480-c321-4a6b-84b4-66da593578ee admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:52.848694 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 913/1819] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 19:18:52 2022] POST /identity/v3/users => generated 300 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 19:18:52.853098 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519df575-91b1-4930-86c5-d39a5040eea9 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:52.863555 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519df575-91b1-4930-86c5-d39a5040eea9 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:52.868635 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85e5879-0618-4967-a319-df97c5a3aea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:52.869237 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85e5879-0618-4967-a319-df97c5a3aea0 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:52.869523 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85e5879-0618-4967-a319-df97c5a3aea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:52.869794 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85e5879-0618-4967-a319-df97c5a3aea0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:52.870466 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85e5879-0618-4967-a319-df97c5a3aea0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:52.872109 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85e5879-0618-4967-a319-df97c5a3aea0 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:52.875853 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-1410d480-c321-4a6b-84b4-66da593578ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b21ad8c810474c67a82f6a285ba1664e'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:52.876527 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-1410d480-c321-4a6b-84b4-66da593578ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b21ad8c810474c67a82f6a285ba1664e'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:52.882165 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 914/1820] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:18:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 19:18:52.885099 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-1410d480-c321-4a6b-84b4-66da593578ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b21ad8c810474c67a82f6a285ba1664e'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:52.886564 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85e5879-0618-4967-a319-df97c5a3aea0 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:52.888233 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-d85e5879-0618-4967-a319-df97c5a3aea0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:52.892663 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-1410d480-c321-4a6b-84b4-66da593578ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b21ad8c810474c67a82f6a285ba1664e'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:52.902684 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85e5879-0618-4967-a319-df97c5a3aea0 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:52.905739 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-1410d480-c321-4a6b-84b4-66da593578ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b21ad8c810474c67a82f6a285ba1664e'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:52.907656 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1117cf-4750-4bce-87c9-50c699ef5c06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:52.908273 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1117cf-4750-4bce-87c9-50c699ef5c06 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:52.908546 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1117cf-4750-4bce-87c9-50c699ef5c06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:52.908832 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1117cf-4750-4bce-87c9-50c699ef5c06 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:52.909479 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1117cf-4750-4bce-87c9-50c699ef5c06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:52.911049 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1117cf-4750-4bce-87c9-50c699ef5c06 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:52.915392 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 907/1821] 10.222.0.61 () {58 vars in 1224 bytes} [Mon Aug 22 19:18:52 2022] DELETE /identity/v3/users/b21ad8c810474c67a82f6a285ba1664e => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 19:18:52.925037 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 915/1822] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:18:52 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 19:18:52.926181 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1410d480-c321-4a6b-84b4-66da593578ee admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:52.959782 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:18:52.998505 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1410d480-c321-4a6b-84b4-66da593578ee admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:53.004035 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d958c814-d78c-4b2c-abb6-546be18603b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:53.004896 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d958c814-d78c-4b2c-abb6-546be18603b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:53.005177 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d958c814-d78c-4b2c-abb6-546be18603b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:53.005445 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d958c814-d78c-4b2c-abb6-546be18603b2 admin admin] PATH_INFO: `/v3/projects/3e11bc01a5184407b3084ce7db135625/users/b7d20a4fee4b40b58c94d0bbf83818b8/roles/74a40bc819cb42eea7e966a4b2a016f1` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:53.021974 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d958c814-d78c-4b2c-abb6-546be18603b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e11bc01a5184407b3084ce7db135625, user_id=b7d20a4fee4b40b58c94d0bbf83818b8, role_id=74a40bc819cb42eea7e966a4b2a016f1)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:53.024884 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d958c814-d78c-4b2c-abb6-546be18603b2 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:53.040419 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 908/1823] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:18:52 2022] PUT /identity/v3/projects/3e11bc01a5184407b3084ce7db135625/users/b7d20a4fee4b40b58c94d0bbf83818b8/roles/74a40bc819cb42eea7e966a4b2a016f1 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:18:53.051727 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a862805d-a71c-451f-97b4-4b4f6512abb2 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:53.052033 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a862805d-a71c-451f-97b4-4b4f6512abb2 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:53.052397 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a862805d-a71c-451f-97b4-4b4f6512abb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:53.083361 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-a862805d-a71c-451f-97b4-4b4f6512abb2 None None] MFA Rules not processed for user `b7d20a4fee4b40b58c94d0bbf83818b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:18:53.084661 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-a862805d-a71c-451f-97b4-4b4f6512abb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:53.167702 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 916/1824] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:18:53 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:18:53.172244 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d958c814-d78c-4b2c-abb6-546be18603b2 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:53.239890 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d958c814-d78c-4b2c-abb6-546be18603b2 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:53.246236 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbaa7fb-299b-44a1-bd84-4601c3f6ac0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:53.246857 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbaa7fb-299b-44a1-bd84-4601c3f6ac0e admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:53.247137 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbaa7fb-299b-44a1-bd84-4601c3f6ac0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:53.247407 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbaa7fb-299b-44a1-bd84-4601c3f6ac0e admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:53.248675 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbaa7fb-299b-44a1-bd84-4601c3f6ac0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:53.250573 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbaa7fb-299b-44a1-bd84-4601c3f6ac0e admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:53.283411 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 909/1825] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 19:18:53 2022] POST /identity/v3/projects => generated 399 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 19:18:53.287397 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a862805d-a71c-451f-97b4-4b4f6512abb2 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:53.299258 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a862805d-a71c-451f-97b4-4b4f6512abb2 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:53.374614 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed5446e-c006-4cc1-a4d2-3cfc9710ab94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:53.375123 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed5446e-c006-4cc1-a4d2-3cfc9710ab94 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:53.375491 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed5446e-c006-4cc1-a4d2-3cfc9710ab94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:53.377899 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed5446e-c006-4cc1-a4d2-3cfc9710ab94 admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:53.378639 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed5446e-c006-4cc1-a4d2-3cfc9710ab94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:53.381060 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed5446e-c006-4cc1-a4d2-3cfc9710ab94 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:53.404497 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:18:53.411579 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 917/1826] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 19:18:53 2022] POST /identity/v3/users => generated 306 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:18:53.415675 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbaa7fb-299b-44a1-bd84-4601c3f6ac0e admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:53.434090 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbaa7fb-299b-44a1-bd84-4601c3f6ac0e admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:53.439986 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d8620f-a711-4428-bfcf-9944e68cc016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:53.440683 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d8620f-a711-4428-bfcf-9944e68cc016 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:53.441036 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d8620f-a711-4428-bfcf-9944e68cc016 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:53.441398 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d8620f-a711-4428-bfcf-9944e68cc016 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:53.442137 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d8620f-a711-4428-bfcf-9944e68cc016 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:53.444454 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d8620f-a711-4428-bfcf-9944e68cc016 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:53.457046 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 910/1827] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:18:53 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 19:18:53.465165 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed5446e-c006-4cc1-a4d2-3cfc9710ab94 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:53.481416 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed5446e-c006-4cc1-a4d2-3cfc9710ab94 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:53.487689 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40286ba-d104-4f63-9f3d-e30bb466f292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:53.489270 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40286ba-d104-4f63-9f3d-e30bb466f292 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:53.489557 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40286ba-d104-4f63-9f3d-e30bb466f292 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:53.490004 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40286ba-d104-4f63-9f3d-e30bb466f292 admin admin] PATH_INFO: `/v3/projects/d986814d4fc8472ea26b4e2163c91989/users/f8d2775757f7426b83f3fcebed35e918/roles/163c30c3b3c246beb849ce176f02d9da` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:53.517557 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d8620f-a711-4428-bfcf-9944e68cc016 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:53.517557 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-24d8620f-a711-4428-bfcf-9944e68cc016 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:53.517758 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40286ba-d104-4f63-9f3d-e30bb466f292 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d986814d4fc8472ea26b4e2163c91989, user_id=f8d2775757f7426b83f3fcebed35e918, role_id=163c30c3b3c246beb849ce176f02d9da)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:53.517758 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40286ba-d104-4f63-9f3d-e30bb466f292 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:53.536077 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 918/1828] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:18:53 2022] PUT /identity/v3/projects/d986814d4fc8472ea26b4e2163c91989/users/f8d2775757f7426b83f3fcebed35e918/roles/163c30c3b3c246beb849ce176f02d9da => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:18:53.539833 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40286ba-d104-4f63-9f3d-e30bb466f292 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:53.543952 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d8620f-a711-4428-bfcf-9944e68cc016 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:53.610447 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40286ba-d104-4f63-9f3d-e30bb466f292 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:53.615704 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816e327a-17e7-4e2f-9043-07294ad0fbb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:53.616314 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816e327a-17e7-4e2f-9043-07294ad0fbb6 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:53.616596 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816e327a-17e7-4e2f-9043-07294ad0fbb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:53.616867 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816e327a-17e7-4e2f-9043-07294ad0fbb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:53.617512 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816e327a-17e7-4e2f-9043-07294ad0fbb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:53.619104 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816e327a-17e7-4e2f-9043-07294ad0fbb6 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:53.629067 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 919/1829] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:18:53 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 19:18:53.630008 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e299b41e-d340-4b44-ba1e-6c8908184502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:53.630653 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e299b41e-d340-4b44-ba1e-6c8908184502 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:53.630935 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e299b41e-d340-4b44-ba1e-6c8908184502 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:53.631208 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e299b41e-d340-4b44-ba1e-6c8908184502 admin admin] PATH_INFO: `/v3/projects/a0189ca307584838bf7ede9fb4843070` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:53.633095 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e299b41e-d340-4b44-ba1e-6c8908184502 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0189ca307584838bf7ede9fb4843070)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:53.633213 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816e327a-17e7-4e2f-9043-07294ad0fbb6 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:53.635584 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e299b41e-d340-4b44-ba1e-6c8908184502 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:53.646289 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816e327a-17e7-4e2f-9043-07294ad0fbb6 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:53.651359 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77561e7-14b7-44a5-8a1d-e6d8f20e6d52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:53.652639 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77561e7-14b7-44a5-8a1d-e6d8f20e6d52 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:53.653056 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77561e7-14b7-44a5-8a1d-e6d8f20e6d52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:53.653571 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77561e7-14b7-44a5-8a1d-e6d8f20e6d52 admin admin] PATH_INFO: `/v3/projects/d986814d4fc8472ea26b4e2163c91989/users/f8d2775757f7426b83f3fcebed35e918/roles/163c30c3b3c246beb849ce176f02d9da` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:53.655862 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77561e7-14b7-44a5-8a1d-e6d8f20e6d52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d986814d4fc8472ea26b4e2163c91989, user_id=f8d2775757f7426b83f3fcebed35e918, role_id=163c30c3b3c246beb849ce176f02d9da)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:53.658692 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77561e7-14b7-44a5-8a1d-e6d8f20e6d52 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:53.676891 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:18:53.685810 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 920/1830] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:18:53 2022] PUT /identity/v3/projects/d986814d4fc8472ea26b4e2163c91989/users/f8d2775757f7426b83f3fcebed35e918/roles/163c30c3b3c246beb849ce176f02d9da => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 19:18:53.687387 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0af198a-a6aa-46ac-a7c9-58bdbaacae9e None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:53.687717 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0af198a-a6aa-46ac-a7c9-58bdbaacae9e None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:53.688184 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0af198a-a6aa-46ac-a7c9-58bdbaacae9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:53.697925 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-e299b41e-d340-4b44-ba1e-6c8908184502 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0189ca307584838bf7ede9fb4843070', 'request_id': 'req-e299b41e-d340-4b44-ba1e-6c8908184502'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:53.705084 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-e299b41e-d340-4b44-ba1e-6c8908184502 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0189ca307584838bf7ede9fb4843070', 'request_id': 'req-e299b41e-d340-4b44-ba1e-6c8908184502'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:53.712761 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-e299b41e-d340-4b44-ba1e-6c8908184502 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0189ca307584838bf7ede9fb4843070', 'request_id': 'req-e299b41e-d340-4b44-ba1e-6c8908184502'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:53.721970 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-f0af198a-a6aa-46ac-a7c9-58bdbaacae9e None None] MFA Rules not processed for user `f8d2775757f7426b83f3fcebed35e918`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:18:53.723203 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-f0af198a-a6aa-46ac-a7c9-58bdbaacae9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:53.725121 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-e299b41e-d340-4b44-ba1e-6c8908184502 admin admin] The token cache is being invalidate because project a0189ca307584838bf7ede9fb4843070 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 19:18:53.725412 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-e299b41e-d340-4b44-ba1e-6c8908184502 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:53.726125 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-e299b41e-d340-4b44-ba1e-6c8908184502 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:18:53.728776 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 911/1831] 10.222.0.61 () {58 vars in 1233 bytes} [Mon Aug 22 19:18:53 2022] DELETE /identity/v3/projects/a0189ca307584838bf7ede9fb4843070 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 19:18:53.733213 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c2f8d1-f3a3-4ca9-8660-fcb655aa84fe None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:53.733493 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c2f8d1-f3a3-4ca9-8660-fcb655aa84fe None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:53.733759 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c2f8d1-f3a3-4ca9-8660-fcb655aa84fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:53.765491 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-81c2f8d1-f3a3-4ca9-8660-fcb655aa84fe None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:18:53.766852 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-81c2f8d1-f3a3-4ca9-8660-fcb655aa84fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:53.809393 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 921/1832] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:18:53 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:18:53.820271 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0af198a-a6aa-46ac-a7c9-58bdbaacae9e None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:53.821683 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-f0af198a-a6aa-46ac-a7c9-58bdbaacae9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:53.835002 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 912/1833] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:18:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 19:18:53.843883 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea598f4e-1c2f-4ab2-86ec-769fc21441d9 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:53.844404 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea598f4e-1c2f-4ab2-86ec-769fc21441d9 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:53.844690 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea598f4e-1c2f-4ab2-86ec-769fc21441d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:53.863988 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-ea598f4e-1c2f-4ab2-86ec-769fc21441d9 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:18:53.865207 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-ea598f4e-1c2f-4ab2-86ec-769fc21441d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:53.868351 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:18:53.874568 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 913/1834] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:18:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 19:18:53.878779 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea598f4e-1c2f-4ab2-86ec-769fc21441d9 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:53.882908 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0af198a-a6aa-46ac-a7c9-58bdbaacae9e None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:53.887260 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3c15b1-9558-4337-b8a8-162f2b8a2523 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e79498873d4ed29a0e19bbea9b9214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:53.887924 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3c15b1-9558-4337-b8a8-162f2b8a2523 service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:53.888182 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3c15b1-9558-4337-b8a8-162f2b8a2523 service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:53.888451 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3c15b1-9558-4337-b8a8-162f2b8a2523 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:53.889379 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea598f4e-1c2f-4ab2-86ec-769fc21441d9 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:53.893980 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb75196-f9c4-4828-a1d7-0f1770a09464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:53.894507 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb75196-f9c4-4828-a1d7-0f1770a09464 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:53.894784 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb75196-f9c4-4828-a1d7-0f1770a09464 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:53.895053 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb75196-f9c4-4828-a1d7-0f1770a09464 admin admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:53.895713 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb75196-f9c4-4828-a1d7-0f1770a09464 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:18:53.896021 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb75196-f9c4-4828-a1d7-0f1770a09464 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:53.897588 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb75196-f9c4-4828-a1d7-0f1770a09464 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:53.897698 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3c15b1-9558-4337-b8a8-162f2b8a2523 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:53.899122 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3c15b1-9558-4337-b8a8-162f2b8a2523 service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:53.908475 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 922/1835] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:18:53 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 19:18:53.909003 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 914/1836] 10.222.0.61 () {58 vars in 1153 bytes} [Mon Aug 22 19:18:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 19:18:53.913354 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3c15b1-9558-4337-b8a8-162f2b8a2523 service cinder] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:53.923203 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3c15b1-9558-4337-b8a8-162f2b8a2523 service cinder] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:53.928569 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b9ed98-d45e-4925-8590-04e9f29732fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:53.929185 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b9ed98-d45e-4925-8590-04e9f29732fa admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:53.929467 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b9ed98-d45e-4925-8590-04e9f29732fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:53.929746 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b9ed98-d45e-4925-8590-04e9f29732fa admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:53.930577 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b9ed98-d45e-4925-8590-04e9f29732fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:53.932516 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b9ed98-d45e-4925-8590-04e9f29732fa admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:53.961049 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 923/1837] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 19:18:53 2022] POST /identity/v3/projects => generated 393 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:18:53.965843 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb75196-f9c4-4828-a1d7-0f1770a09464 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:53.981907 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb75196-f9c4-4828-a1d7-0f1770a09464 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:53.989430 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b9ed98-d45e-4925-8590-04e9f29732fa admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:54.034201 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ac46a2-974b-4312-aaf2-2aaf8f425131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:54.034815 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ac46a2-974b-4312-aaf2-2aaf8f425131 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:54.035103 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ac46a2-974b-4312-aaf2-2aaf8f425131 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:54.035378 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ac46a2-974b-4312-aaf2-2aaf8f425131 admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:54.036039 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ac46a2-974b-4312-aaf2-2aaf8f425131 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:54.037957 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ac46a2-974b-4312-aaf2-2aaf8f425131 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:54.047249 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b9ed98-d45e-4925-8590-04e9f29732fa admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:54.051326 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5219ac46-c9ec-43c0-a6a5-d83297a488d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e79498873d4ed29a0e19bbea9b9214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:54.052128 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5219ac46-c9ec-43c0-a6a5-d83297a488d5 service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:54.052493 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5219ac46-c9ec-43c0-a6a5-d83297a488d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:54.052718 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5219ac46-c9ec-43c0-a6a5-d83297a488d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:54.054570 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-5219ac46-c9ec-43c0-a6a5-d83297a488d5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:54.059935 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 915/1838] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 19:18:53 2022] POST /identity/v3/users => generated 297 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:18:54.064072 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ac46a2-974b-4312-aaf2-2aaf8f425131 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:54.074477 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ac46a2-974b-4312-aaf2-2aaf8f425131 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:54.079242 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd2c113-d54c-4d70-a606-86a7c45469a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:54.079893 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd2c113-d54c-4d70-a606-86a7c45469a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:54.080174 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd2c113-d54c-4d70-a606-86a7c45469a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:54.080597 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd2c113-d54c-4d70-a606-86a7c45469a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:54.081102 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd2c113-d54c-4d70-a606-86a7c45469a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:54.082710 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd2c113-d54c-4d70-a606-86a7c45469a9 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:54.093688 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 916/1839] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:18:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:18:54.097934 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd2c113-d54c-4d70-a606-86a7c45469a9 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:54.108371 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd2c113-d54c-4d70-a606-86a7c45469a9 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:54.113305 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a11e94-534c-4acb-8161-ebe5dcf78bab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:54.113999 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a11e94-534c-4acb-8161-ebe5dcf78bab admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:54.114313 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a11e94-534c-4acb-8161-ebe5dcf78bab admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:54.114586 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a11e94-534c-4acb-8161-ebe5dcf78bab admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:54.115341 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a11e94-534c-4acb-8161-ebe5dcf78bab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:54.116997 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a11e94-534c-4acb-8161-ebe5dcf78bab admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:54.118041 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5219ac46-c9ec-43c0-a6a5-d83297a488d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:54.119480 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5219ac46-c9ec-43c0-a6a5-d83297a488d5 service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:54.128049 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 917/1840] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:18:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 19:18:54.129835 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 924/1841] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:18:53 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:18:54.132105 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a11e94-534c-4acb-8161-ebe5dcf78bab admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:54.142192 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a11e94-534c-4acb-8161-ebe5dcf78bab admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:54.146986 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98371cf-30e8-4265-947b-4765467de6fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:54.148103 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98371cf-30e8-4265-947b-4765467de6fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:54.148203 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98371cf-30e8-4265-947b-4765467de6fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:54.148447 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98371cf-30e8-4265-947b-4765467de6fe admin admin] PATH_INFO: `/v3/projects/b14911c5cb6d484c882cd05c2d45ab20/users/d36b7c580a664c2daff8cc2437654cc8/roles/74a40bc819cb42eea7e966a4b2a016f1` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:54.150771 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:18:54.169526 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98371cf-30e8-4265-947b-4765467de6fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=b14911c5cb6d484c882cd05c2d45ab20, user_id=d36b7c580a664c2daff8cc2437654cc8, role_id=74a40bc819cb42eea7e966a4b2a016f1)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:54.172320 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98371cf-30e8-4265-947b-4765467de6fe admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:54.188747 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 918/1842] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:18:54 2022] PUT /identity/v3/projects/b14911c5cb6d484c882cd05c2d45ab20/users/d36b7c580a664c2daff8cc2437654cc8/roles/74a40bc819cb42eea7e966a4b2a016f1 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:18:54.199360 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0642fdc1-b142-4207-8422-f82601b8a0ea None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:54.199673 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0642fdc1-b142-4207-8422-f82601b8a0ea None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:54.199948 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0642fdc1-b142-4207-8422-f82601b8a0ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:54.228177 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-0642fdc1-b142-4207-8422-f82601b8a0ea None None] MFA Rules not processed for user `d36b7c580a664c2daff8cc2437654cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:18:54.230092 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-0642fdc1-b142-4207-8422-f82601b8a0ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:54.299393 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 925/1843] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:18:54 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:18:54.303967 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98371cf-30e8-4265-947b-4765467de6fe admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:54.372892 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98371cf-30e8-4265-947b-4765467de6fe admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:54.380174 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2c9335-d054-4bd5-8e7c-9efa3571c885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:54.380970 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2c9335-d054-4bd5-8e7c-9efa3571c885 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:54.381535 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2c9335-d054-4bd5-8e7c-9efa3571c885 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:54.382006 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2c9335-d054-4bd5-8e7c-9efa3571c885 admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:54.382668 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2c9335-d054-4bd5-8e7c-9efa3571c885 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:54.384515 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2c9335-d054-4bd5-8e7c-9efa3571c885 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:54.430064 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 919/1844] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 19:18:54 2022] POST /identity/v3/projects => generated 391 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 19:18:54.435952 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0642fdc1-b142-4207-8422-f82601b8a0ea None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:54.446100 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0642fdc1-b142-4207-8422-f82601b8a0ea None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:54.451037 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:18:54.510751 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4757b4b6-253a-45f2-be67-e28519be6e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:54.511391 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4757b4b6-253a-45f2-be67-e28519be6e97 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:54.511687 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4757b4b6-253a-45f2-be67-e28519be6e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:54.511962 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4757b4b6-253a-45f2-be67-e28519be6e97 admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:54.512845 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4757b4b6-253a-45f2-be67-e28519be6e97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:54.514766 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4757b4b6-253a-45f2-be67-e28519be6e97 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:54.538708 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 926/1845] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 19:18:54 2022] POST /identity/v3/users => generated 302 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:18:54.542957 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2c9335-d054-4bd5-8e7c-9efa3571c885 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:54.553901 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2c9335-d054-4bd5-8e7c-9efa3571c885 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:54.559025 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc83c29-b59b-4865-9c20-4f3a6fe7e2f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:54.559622 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc83c29-b59b-4865-9c20-4f3a6fe7e2f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:54.559972 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc83c29-b59b-4865-9c20-4f3a6fe7e2f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:54.560258 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc83c29-b59b-4865-9c20-4f3a6fe7e2f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:54.560911 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc83c29-b59b-4865-9c20-4f3a6fe7e2f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:54.562568 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc83c29-b59b-4865-9c20-4f3a6fe7e2f1 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:54.575156 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 920/1846] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:18:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:18:54.579414 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4757b4b6-253a-45f2-be67-e28519be6e97 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:54.590464 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4757b4b6-253a-45f2-be67-e28519be6e97 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:54.595228 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4b8009-f183-41ca-8b7d-f8aa7f101245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:54.595924 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4b8009-f183-41ca-8b7d-f8aa7f101245 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:54.596200 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4b8009-f183-41ca-8b7d-f8aa7f101245 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:54.596467 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4b8009-f183-41ca-8b7d-f8aa7f101245 admin admin] PATH_INFO: `/v3/projects/f486a16a9ff3429cbac9f4426300d048/users/4f6a2b198d0b48e5a0004e799ddd017e/roles/163c30c3b3c246beb849ce176f02d9da` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:54.612782 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4b8009-f183-41ca-8b7d-f8aa7f101245 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f486a16a9ff3429cbac9f4426300d048, user_id=4f6a2b198d0b48e5a0004e799ddd017e, role_id=163c30c3b3c246beb849ce176f02d9da)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:54.615911 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4b8009-f183-41ca-8b7d-f8aa7f101245 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:54.629662 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 927/1847] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:18:54 2022] PUT /identity/v3/projects/f486a16a9ff3429cbac9f4426300d048/users/4f6a2b198d0b48e5a0004e799ddd017e/roles/163c30c3b3c246beb849ce176f02d9da => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:18:54.633645 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc83c29-b59b-4865-9c20-4f3a6fe7e2f1 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:54.645072 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc83c29-b59b-4865-9c20-4f3a6fe7e2f1 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:54.712100 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ac9460-7836-4c68-aec4-12be3849d5a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:54.712746 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ac9460-7836-4c68-aec4-12be3849d5a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:54.713030 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ac9460-7836-4c68-aec4-12be3849d5a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:54.713295 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ac9460-7836-4c68-aec4-12be3849d5a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:54.714463 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ac9460-7836-4c68-aec4-12be3849d5a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:54.716209 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ac9460-7836-4c68-aec4-12be3849d5a9 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:54.738896 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 921/1848] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:18:54 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:18:54.743045 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4b8009-f183-41ca-8b7d-f8aa7f101245 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:54.753595 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4b8009-f183-41ca-8b7d-f8aa7f101245 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:54.758315 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa14ec8-9ad2-4b54-9c79-e70311480f42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:54.758974 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa14ec8-9ad2-4b54-9c79-e70311480f42 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:54.759258 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa14ec8-9ad2-4b54-9c79-e70311480f42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:54.759533 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa14ec8-9ad2-4b54-9c79-e70311480f42 admin admin] PATH_INFO: `/v3/projects/f486a16a9ff3429cbac9f4426300d048/users/4f6a2b198d0b48e5a0004e799ddd017e/roles/163c30c3b3c246beb849ce176f02d9da` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:54.761555 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa14ec8-9ad2-4b54-9c79-e70311480f42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f486a16a9ff3429cbac9f4426300d048, user_id=4f6a2b198d0b48e5a0004e799ddd017e, role_id=163c30c3b3c246beb849ce176f02d9da)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:54.764219 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa14ec8-9ad2-4b54-9c79-e70311480f42 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:54.775748 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 928/1849] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:18:54 2022] PUT /identity/v3/projects/f486a16a9ff3429cbac9f4426300d048/users/4f6a2b198d0b48e5a0004e799ddd017e/roles/163c30c3b3c246beb849ce176f02d9da => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:18:54.787286 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba22618e-d3b8-4e0d-b58a-efb249fc14bb None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:54.787571 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba22618e-d3b8-4e0d-b58a-efb249fc14bb None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:54.787887 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba22618e-d3b8-4e0d-b58a-efb249fc14bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:54.815463 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-ba22618e-d3b8-4e0d-b58a-efb249fc14bb None None] MFA Rules not processed for user `4f6a2b198d0b48e5a0004e799ddd017e`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:18:54.816722 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-ba22618e-d3b8-4e0d-b58a-efb249fc14bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:54.860037 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:18:54.888615 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 922/1850] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:18:54 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:18:54.899158 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa14ec8-9ad2-4b54-9c79-e70311480f42 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:54.900997 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-5fa14ec8-9ad2-4b54-9c79-e70311480f42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:18:54.960203 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa14ec8-9ad2-4b54-9c79-e70311480f42 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:54.963798 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99154b62-0d6f-406a-b194-b0ba8af3ab72 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e79498873d4ed29a0e19bbea9b9214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:54.964975 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99154b62-0d6f-406a-b194-b0ba8af3ab72 service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:54.965248 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99154b62-0d6f-406a-b194-b0ba8af3ab72 service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:54.965513 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99154b62-0d6f-406a-b194-b0ba8af3ab72 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:54.975253 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99154b62-0d6f-406a-b194-b0ba8af3ab72 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:54.976977 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99154b62-0d6f-406a-b194-b0ba8af3ab72 service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:54.986831 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 929/1851] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:18:54 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:18:55.040279 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba22618e-d3b8-4e0d-b58a-efb249fc14bb None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:18:55.051387 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba22618e-d3b8-4e0d-b58a-efb249fc14bb None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:18:55.054971 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0851e2e-5862-49b8-9d54-e558823635eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e79498873d4ed29a0e19bbea9b9214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:18:55.055564 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0851e2e-5862-49b8-9d54-e558823635eb service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:18:55.055872 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0851e2e-5862-49b8-9d54-e558823635eb service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:18:55.056142 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0851e2e-5862-49b8-9d54-e558823635eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:18:55.112919 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0851e2e-5862-49b8-9d54-e558823635eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:18:55.114350 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0851e2e-5862-49b8-9d54-e558823635eb service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:18:55.123423 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 923/1852] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:18:55 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:19:01.402125 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99154b62-0d6f-406a-b194-b0ba8af3ab72 service cinder] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:01.410543 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99154b62-0d6f-406a-b194-b0ba8af3ab72 service cinder] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:01.414168 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a4ed5d-88be-401e-a318-be7e41b3579c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e79498873d4ed29a0e19bbea9b9214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:01.414761 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a4ed5d-88be-401e-a318-be7e41b3579c service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:01.415043 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a4ed5d-88be-401e-a318-be7e41b3579c service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:01.415311 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a4ed5d-88be-401e-a318-be7e41b3579c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:01.417057 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-26a4ed5d-88be-401e-a318-be7e41b3579c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:01.478283 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a4ed5d-88be-401e-a318-be7e41b3579c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:01.479730 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a4ed5d-88be-401e-a318-be7e41b3579c service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:01.483638 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:19:01.491835 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 930/1853] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:19:01 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:19:05.119326 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=38 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:19:09.141170 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0851e2e-5862-49b8-9d54-e558823635eb service cinder] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:09.234249 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0851e2e-5862-49b8-9d54-e558823635eb service cinder] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:09.239094 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1fd108-fb0c-4dfa-af8d-f9e825edbeac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:09.239825 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1fd108-fb0c-4dfa-af8d-f9e825edbeac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:09.240109 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1fd108-fb0c-4dfa-af8d-f9e825edbeac admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:09.240382 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1fd108-fb0c-4dfa-af8d-f9e825edbeac admin admin] PATH_INFO: `/v3/users/d36b7c580a664c2daff8cc2437654cc8` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:09.241519 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1fd108-fb0c-4dfa-af8d-f9e825edbeac admin admin] RBAC: Authorizing `identity:delete_user(user_id=d36b7c580a664c2daff8cc2437654cc8)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:09.243418 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1fd108-fb0c-4dfa-af8d-f9e825edbeac admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:09.299551 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-df1fd108-fb0c-4dfa-af8d-f9e825edbeac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd36b7c580a664c2daff8cc2437654cc8'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:09.300415 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-df1fd108-fb0c-4dfa-af8d-f9e825edbeac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd36b7c580a664c2daff8cc2437654cc8'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:09.308803 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-df1fd108-fb0c-4dfa-af8d-f9e825edbeac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd36b7c580a664c2daff8cc2437654cc8'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:09.319934 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-df1fd108-fb0c-4dfa-af8d-f9e825edbeac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd36b7c580a664c2daff8cc2437654cc8'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:09.334169 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-df1fd108-fb0c-4dfa-af8d-f9e825edbeac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd36b7c580a664c2daff8cc2437654cc8'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:09.346194 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 924/1854] 10.222.0.61 () {58 vars in 1224 bytes} [Mon Aug 22 19:19:09 2022] DELETE /identity/v3/users/d36b7c580a664c2daff8cc2437654cc8 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:19:09.350865 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476da383-192f-405e-a077-5803a2439d79 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:09.351339 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476da383-192f-405e-a077-5803a2439d79 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:09.351518 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476da383-192f-405e-a077-5803a2439d79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:09.370968 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-476da383-192f-405e-a077-5803a2439d79 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:19:09.372227 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-476da383-192f-405e-a077-5803a2439d79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:09.432106 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 931/1855] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:19:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:19:09.441764 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1fd108-fb0c-4dfa-af8d-f9e825edbeac admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:09.443179 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-df1fd108-fb0c-4dfa-af8d-f9e825edbeac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:09.501829 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1fd108-fb0c-4dfa-af8d-f9e825edbeac admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:09.505628 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194ca37d-44fd-418a-ba6f-442ad554e26e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:09.506238 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194ca37d-44fd-418a-ba6f-442ad554e26e service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:09.506511 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194ca37d-44fd-418a-ba6f-442ad554e26e service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:09.506802 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194ca37d-44fd-418a-ba6f-442ad554e26e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:09.517324 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194ca37d-44fd-418a-ba6f-442ad554e26e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:09.518667 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194ca37d-44fd-418a-ba6f-442ad554e26e service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:09.528467 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 925/1856] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:19:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:19:09.809463 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476da383-192f-405e-a077-5803a2439d79 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:09.810986 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-476da383-192f-405e-a077-5803a2439d79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:09.831943 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476da383-192f-405e-a077-5803a2439d79 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:09.838659 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62015e3b-33a1-4812-bd8c-7d5f0e709033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:09.839607 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62015e3b-33a1-4812-bd8c-7d5f0e709033 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:09.840207 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62015e3b-33a1-4812-bd8c-7d5f0e709033 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:09.840556 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62015e3b-33a1-4812-bd8c-7d5f0e709033 admin admin] PATH_INFO: `/v3/projects/b14911c5cb6d484c882cd05c2d45ab20` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:09.842224 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62015e3b-33a1-4812-bd8c-7d5f0e709033 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b14911c5cb6d484c882cd05c2d45ab20)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:09.844345 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62015e3b-33a1-4812-bd8c-7d5f0e709033 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:09.894735 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-62015e3b-33a1-4812-bd8c-7d5f0e709033 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b14911c5cb6d484c882cd05c2d45ab20', 'request_id': 'req-62015e3b-33a1-4812-bd8c-7d5f0e709033'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:09.904041 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-62015e3b-33a1-4812-bd8c-7d5f0e709033 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b14911c5cb6d484c882cd05c2d45ab20', 'request_id': 'req-62015e3b-33a1-4812-bd8c-7d5f0e709033'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:09.910606 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-62015e3b-33a1-4812-bd8c-7d5f0e709033 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b14911c5cb6d484c882cd05c2d45ab20', 'request_id': 'req-62015e3b-33a1-4812-bd8c-7d5f0e709033'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:09.922411 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-62015e3b-33a1-4812-bd8c-7d5f0e709033 admin admin] The token cache is being invalidate because project b14911c5cb6d484c882cd05c2d45ab20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 19:19:09.922702 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-62015e3b-33a1-4812-bd8c-7d5f0e709033 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:09.923355 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-62015e3b-33a1-4812-bd8c-7d5f0e709033 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:09.925587 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 932/1857] 10.222.0.61 () {58 vars in 1233 bytes} [Mon Aug 22 19:19:09 2022] DELETE /identity/v3/projects/b14911c5cb6d484c882cd05c2d45ab20 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:19:09.929690 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194ca37d-44fd-418a-ba6f-442ad554e26e service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:09.940285 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194ca37d-44fd-418a-ba6f-442ad554e26e service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:09.958608 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:19:09.967152 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62015e3b-33a1-4812-bd8c-7d5f0e709033 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:09.968743 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-62015e3b-33a1-4812-bd8c-7d5f0e709033 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:09.999957 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc861a5f-c237-490b-b054-61b62c9d5dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:10.000692 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc861a5f-c237-490b-b054-61b62c9d5dd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:10.001298 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc861a5f-c237-490b-b054-61b62c9d5dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:10.001298 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc861a5f-c237-490b-b054-61b62c9d5dd5 admin admin] PATH_INFO: `/v3/users/4f6a2b198d0b48e5a0004e799ddd017e` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:10.002520 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc861a5f-c237-490b-b054-61b62c9d5dd5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f6a2b198d0b48e5a0004e799ddd017e)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:10.004541 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc861a5f-c237-490b-b054-61b62c9d5dd5 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:10.037826 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62015e3b-33a1-4812-bd8c-7d5f0e709033 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:10.043729 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d3e558-14be-4d79-9daa-b8a7a661abf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:10.044380 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d3e558-14be-4d79-9daa-b8a7a661abf5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:10.044868 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d3e558-14be-4d79-9daa-b8a7a661abf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:10.045152 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d3e558-14be-4d79-9daa-b8a7a661abf5 admin admin] PATH_INFO: `/v3/users/b7d20a4fee4b40b58c94d0bbf83818b8` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:10.046538 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d3e558-14be-4d79-9daa-b8a7a661abf5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7d20a4fee4b40b58c94d0bbf83818b8)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:10.048608 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d3e558-14be-4d79-9daa-b8a7a661abf5 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:10.060216 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-bc861a5f-c237-490b-b054-61b62c9d5dd5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f6a2b198d0b48e5a0004e799ddd017e'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:10.060882 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-bc861a5f-c237-490b-b054-61b62c9d5dd5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f6a2b198d0b48e5a0004e799ddd017e'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:10.069046 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-bc861a5f-c237-490b-b054-61b62c9d5dd5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f6a2b198d0b48e5a0004e799ddd017e'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:10.074855 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:19:10.077979 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-bc861a5f-c237-490b-b054-61b62c9d5dd5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f6a2b198d0b48e5a0004e799ddd017e'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:10.098938 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-bc861a5f-c237-490b-b054-61b62c9d5dd5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f6a2b198d0b48e5a0004e799ddd017e'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:10.120734 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-f2d3e558-14be-4d79-9daa-b8a7a661abf5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7d20a4fee4b40b58c94d0bbf83818b8'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:10.122075 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 926/1858] 10.222.0.61 () {58 vars in 1224 bytes} [Mon Aug 22 19:19:09 2022] DELETE /identity/v3/users/4f6a2b198d0b48e5a0004e799ddd017e => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:19:10.122201 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-f2d3e558-14be-4d79-9daa-b8a7a661abf5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7d20a4fee4b40b58c94d0bbf83818b8'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:10.139023 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-f2d3e558-14be-4d79-9daa-b8a7a661abf5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7d20a4fee4b40b58c94d0bbf83818b8'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:10.155621 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-f2d3e558-14be-4d79-9daa-b8a7a661abf5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7d20a4fee4b40b58c94d0bbf83818b8'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:10.181625 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-f2d3e558-14be-4d79-9daa-b8a7a661abf5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7d20a4fee4b40b58c94d0bbf83818b8'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:10.197974 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 933/1859] 10.222.0.61 () {58 vars in 1224 bytes} [Mon Aug 22 19:19:09 2022] DELETE /identity/v3/users/b7d20a4fee4b40b58c94d0bbf83818b8 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 19:19:10.200476 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad64dc07-9075-4eaf-90b6-02704b29e7f0 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:10.201283 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad64dc07-9075-4eaf-90b6-02704b29e7f0 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:10.201717 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad64dc07-9075-4eaf-90b6-02704b29e7f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:10.232853 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-ad64dc07-9075-4eaf-90b6-02704b29e7f0 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:19:10.234673 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-ad64dc07-9075-4eaf-90b6-02704b29e7f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:10.358678 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 927/1860] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:19:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:19:10.369666 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d3e558-14be-4d79-9daa-b8a7a661abf5 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:10.374031 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-f2d3e558-14be-4d79-9daa-b8a7a661abf5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:10.453578 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3e81b8-507f-4d2c-813c-a01055fac65f None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:10.453885 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3e81b8-507f-4d2c-813c-a01055fac65f None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:10.454158 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3e81b8-507f-4d2c-813c-a01055fac65f None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:10.455901 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 928/1861] 10.222.0.61 () {54 vars in 921 bytes} [Mon Aug 22 19:19:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:19:10.470867 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a613d4a-1430-4971-b651-91536d4219d4 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:10.471151 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a613d4a-1430-4971-b651-91536d4219d4 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:10.471523 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a613d4a-1430-4971-b651-91536d4219d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:10.559613 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-8a613d4a-1430-4971-b651-91536d4219d4 None None] MFA Rules not processed for user `e7f43e2756b746a6825ca1a06b7d7c91`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:19:10.561774 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-8a613d4a-1430-4971-b651-91536d4219d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:10.579480 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d3e558-14be-4d79-9daa-b8a7a661abf5 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:10.591692 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a173ff08-b52b-4f35-bfb8-44be5b0feb2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:10.592335 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a173ff08-b52b-4f35-bfb8-44be5b0feb2c service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:10.592614 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a173ff08-b52b-4f35-bfb8-44be5b0feb2c service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:10.592886 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a173ff08-b52b-4f35-bfb8-44be5b0feb2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:10.609332 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a173ff08-b52b-4f35-bfb8-44be5b0feb2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:10.611747 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a173ff08-b52b-4f35-bfb8-44be5b0feb2c service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:10.627725 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 934/1862] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:19:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:19:10.631246 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:19:10.667914 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 929/1863] 10.222.0.61 () {58 vars in 1018 bytes} [Mon Aug 22 19:19:10 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 19:19:10.680261 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a173ff08-b52b-4f35-bfb8-44be5b0feb2c service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:10.682052 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-a173ff08-b52b-4f35-bfb8-44be5b0feb2c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:10.703528 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a173ff08-b52b-4f35-bfb8-44be5b0feb2c service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:10.710459 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c147f7-b871-4138-b2ed-d8ed5f0c53d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f43e2756b746a6825ca1a06b7d7c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:10.711064 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c147f7-b871-4138-b2ed-d8ed5f0c53d0 service nova] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:10.711347 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c147f7-b871-4138-b2ed-d8ed5f0c53d0 service nova] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:10.711622 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c147f7-b871-4138-b2ed-d8ed5f0c53d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:10.723136 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c147f7-b871-4138-b2ed-d8ed5f0c53d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:10.724997 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c147f7-b871-4138-b2ed-d8ed5f0c53d0 service nova] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:10.738521 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 935/1864] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:19:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:19:10.830205 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a613d4a-1430-4971-b651-91536d4219d4 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:10.831851 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-8a613d4a-1430-4971-b651-91536d4219d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:10.848406 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a613d4a-1430-4971-b651-91536d4219d4 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:10.852714 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6e8e0a-6dc5-4e81-8f8a-a636206bc73d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:10.853435 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6e8e0a-6dc5-4e81-8f8a-a636206bc73d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:10.853782 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6e8e0a-6dc5-4e81-8f8a-a636206bc73d admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:10.854144 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6e8e0a-6dc5-4e81-8f8a-a636206bc73d admin admin] PATH_INFO: `/v3/projects/f486a16a9ff3429cbac9f4426300d048` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:10.855402 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6e8e0a-6dc5-4e81-8f8a-a636206bc73d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f486a16a9ff3429cbac9f4426300d048)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:10.857279 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6e8e0a-6dc5-4e81-8f8a-a636206bc73d admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:10.917540 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-7d6e8e0a-6dc5-4e81-8f8a-a636206bc73d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f486a16a9ff3429cbac9f4426300d048', 'request_id': 'req-7d6e8e0a-6dc5-4e81-8f8a-a636206bc73d'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:10.927408 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-7d6e8e0a-6dc5-4e81-8f8a-a636206bc73d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f486a16a9ff3429cbac9f4426300d048', 'request_id': 'req-7d6e8e0a-6dc5-4e81-8f8a-a636206bc73d'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:10.942560 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-7d6e8e0a-6dc5-4e81-8f8a-a636206bc73d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f486a16a9ff3429cbac9f4426300d048', 'request_id': 'req-7d6e8e0a-6dc5-4e81-8f8a-a636206bc73d'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:10.957230 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-7d6e8e0a-6dc5-4e81-8f8a-a636206bc73d admin admin] The token cache is being invalidate because project f486a16a9ff3429cbac9f4426300d048 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 19:19:10.957860 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-7d6e8e0a-6dc5-4e81-8f8a-a636206bc73d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:10.959075 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-7d6e8e0a-6dc5-4e81-8f8a-a636206bc73d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:10.962107 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 930/1865] 10.222.0.61 () {58 vars in 1233 bytes} [Mon Aug 22 19:19:10 2022] DELETE /identity/v3/projects/f486a16a9ff3429cbac9f4426300d048 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:19:10.968826 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226e285b-5bd9-4e61-b1c2-b46674f2181c None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:10.969117 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226e285b-5bd9-4e61-b1c2-b46674f2181c None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:10.969393 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226e285b-5bd9-4e61-b1c2-b46674f2181c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:10.998115 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-226e285b-5bd9-4e61-b1c2-b46674f2181c None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:19:10.999635 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-226e285b-5bd9-4e61-b1c2-b46674f2181c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:11.064923 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6e8e0a-6dc5-4e81-8f8a-a636206bc73d admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:11.066377 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-7d6e8e0a-6dc5-4e81-8f8a-a636206bc73d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:11.073010 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 936/1866] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:19:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:19:11.080713 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6e8e0a-6dc5-4e81-8f8a-a636206bc73d admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:11.081294 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7221b622-9f6b-4206-a335-68a6744d93f8 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:11.081661 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7221b622-9f6b-4206-a335-68a6744d93f8 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:11.082029 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7221b622-9f6b-4206-a335-68a6744d93f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:11.086536 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6594227-0ff0-4fc4-835e-685c65eab425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:11.087245 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6594227-0ff0-4fc4-835e-685c65eab425 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:11.087602 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6594227-0ff0-4fc4-835e-685c65eab425 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:11.088060 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6594227-0ff0-4fc4-835e-685c65eab425 admin admin] PATH_INFO: `/v3/projects/3e11bc01a5184407b3084ce7db135625` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:11.089295 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6594227-0ff0-4fc4-835e-685c65eab425 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e11bc01a5184407b3084ce7db135625)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:11.092336 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6594227-0ff0-4fc4-835e-685c65eab425 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:11.103041 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-7221b622-9f6b-4206-a335-68a6744d93f8 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:19:11.104393 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-7221b622-9f6b-4206-a335-68a6744d93f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:11.112704 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 937/1867] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:19:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:19:11.116765 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7221b622-9f6b-4206-a335-68a6744d93f8 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:11.128786 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7221b622-9f6b-4206-a335-68a6744d93f8 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:11.133409 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec33cb4-65e8-415b-b6bc-b3700a30d1f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:11.134003 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec33cb4-65e8-415b-b6bc-b3700a30d1f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:11.134276 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec33cb4-65e8-415b-b6bc-b3700a30d1f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:11.134545 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec33cb4-65e8-415b-b6bc-b3700a30d1f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:11.135178 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec33cb4-65e8-415b-b6bc-b3700a30d1f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:19:11.135480 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec33cb4-65e8-415b-b6bc-b3700a30d1f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:11.137752 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec33cb4-65e8-415b-b6bc-b3700a30d1f3 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:11.149687 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 938/1868] 10.222.0.61 () {58 vars in 1153 bytes} [Mon Aug 22 19:19:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 19:19:11.149955 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-c6594227-0ff0-4fc4-835e-685c65eab425 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e11bc01a5184407b3084ce7db135625', 'request_id': 'req-c6594227-0ff0-4fc4-835e-685c65eab425'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:11.154022 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec33cb4-65e8-415b-b6bc-b3700a30d1f3 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:11.160180 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-c6594227-0ff0-4fc4-835e-685c65eab425 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e11bc01a5184407b3084ce7db135625', 'request_id': 'req-c6594227-0ff0-4fc4-835e-685c65eab425'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:11.164879 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec33cb4-65e8-415b-b6bc-b3700a30d1f3 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:11.166567 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-c6594227-0ff0-4fc4-835e-685c65eab425 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e11bc01a5184407b3084ce7db135625', 'request_id': 'req-c6594227-0ff0-4fc4-835e-685c65eab425'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:11.173180 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:19:11.177625 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-c6594227-0ff0-4fc4-835e-685c65eab425 admin admin] The token cache is being invalidate because project 3e11bc01a5184407b3084ce7db135625 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 19:19:11.177916 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-c6594227-0ff0-4fc4-835e-685c65eab425 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:11.178581 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-c6594227-0ff0-4fc4-835e-685c65eab425 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:11.180275 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 931/1869] 10.222.0.61 () {58 vars in 1233 bytes} [Mon Aug 22 19:19:11 2022] DELETE /identity/v3/projects/3e11bc01a5184407b3084ce7db135625 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 19:19:11.184639 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6594227-0ff0-4fc4-835e-685c65eab425 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:11.185961 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-c6594227-0ff0-4fc4-835e-685c65eab425 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:11.227926 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626ad708-e3a7-4fa7-8fc9-d375e90f3972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:11.228549 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626ad708-e3a7-4fa7-8fc9-d375e90f3972 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:11.228778 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:19:11.229171 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626ad708-e3a7-4fa7-8fc9-d375e90f3972 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:11.229171 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626ad708-e3a7-4fa7-8fc9-d375e90f3972 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:11.229748 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626ad708-e3a7-4fa7-8fc9-d375e90f3972 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:11.231709 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626ad708-e3a7-4fa7-8fc9-d375e90f3972 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:11.255145 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6594227-0ff0-4fc4-835e-685c65eab425 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:11.261404 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff35f10e-3f65-47a4-91ad-9517e4ea3d8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:11.262050 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff35f10e-3f65-47a4-91ad-9517e4ea3d8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:11.262327 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff35f10e-3f65-47a4-91ad-9517e4ea3d8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:11.262499 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 939/1870] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 19:19:11 2022] POST /identity/v3/projects => generated 377 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 19:19:11.262616 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff35f10e-3f65-47a4-91ad-9517e4ea3d8c admin admin] PATH_INFO: `/v3/users/f8d2775757f7426b83f3fcebed35e918` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:11.263770 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff35f10e-3f65-47a4-91ad-9517e4ea3d8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8d2775757f7426b83f3fcebed35e918)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:11.265733 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff35f10e-3f65-47a4-91ad-9517e4ea3d8c admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:11.266586 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626ad708-e3a7-4fa7-8fc9-d375e90f3972 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:11.268307 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-626ad708-e3a7-4fa7-8fc9-d375e90f3972 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:11.318291 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-ff35f10e-3f65-47a4-91ad-9517e4ea3d8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8d2775757f7426b83f3fcebed35e918'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:11.318975 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-ff35f10e-3f65-47a4-91ad-9517e4ea3d8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8d2775757f7426b83f3fcebed35e918'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:11.327896 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-ff35f10e-3f65-47a4-91ad-9517e4ea3d8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8d2775757f7426b83f3fcebed35e918'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:11.336835 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-ff35f10e-3f65-47a4-91ad-9517e4ea3d8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8d2775757f7426b83f3fcebed35e918'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:11.346770 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626ad708-e3a7-4fa7-8fc9-d375e90f3972 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:11.352738 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38112071-9dcc-433c-89d0-78782c96094c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:11.353532 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38112071-9dcc-433c-89d0-78782c96094c admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:11.353755 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38112071-9dcc-433c-89d0-78782c96094c admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:11.354035 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38112071-9dcc-433c-89d0-78782c96094c admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:11.354940 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-ff35f10e-3f65-47a4-91ad-9517e4ea3d8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8d2775757f7426b83f3fcebed35e918'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:11.355230 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38112071-9dcc-433c-89d0-78782c96094c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:11.358604 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38112071-9dcc-433c-89d0-78782c96094c admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:11.364177 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 932/1871] 10.222.0.61 () {58 vars in 1224 bytes} [Mon Aug 22 19:19:11 2022] DELETE /identity/v3/users/f8d2775757f7426b83f3fcebed35e918 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 19:19:11.387440 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 940/1872] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 19:19:11 2022] POST /identity/v3/users => generated 289 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:19:11.391383 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff35f10e-3f65-47a4-91ad-9517e4ea3d8c admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:11.403668 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-ff35f10e-3f65-47a4-91ad-9517e4ea3d8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:11.460201 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff35f10e-3f65-47a4-91ad-9517e4ea3d8c admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:11.464988 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f52b81-d6cc-4b84-b507-b5bb834fd19e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:11.465600 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f52b81-d6cc-4b84-b507-b5bb834fd19e admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:11.465882 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f52b81-d6cc-4b84-b507-b5bb834fd19e admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:11.466157 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f52b81-d6cc-4b84-b507-b5bb834fd19e admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:11.466895 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f52b81-d6cc-4b84-b507-b5bb834fd19e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:11.468613 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f52b81-d6cc-4b84-b507-b5bb834fd19e admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:11.479082 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 933/1873] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:19:11 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 19:19:11.483599 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38112071-9dcc-433c-89d0-78782c96094c admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:11.494242 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38112071-9dcc-433c-89d0-78782c96094c admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:11.499857 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dd4b85-be09-4b3a-b657-c23745b7bf7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:11.500473 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dd4b85-be09-4b3a-b657-c23745b7bf7d admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:11.500756 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dd4b85-be09-4b3a-b657-c23745b7bf7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:11.501030 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dd4b85-be09-4b3a-b657-c23745b7bf7d admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:11.501714 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dd4b85-be09-4b3a-b657-c23745b7bf7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:11.503324 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dd4b85-be09-4b3a-b657-c23745b7bf7d admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:11.516164 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 941/1874] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:19:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:19:11.518110 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f52b81-d6cc-4b84-b507-b5bb834fd19e admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:11.528610 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f52b81-d6cc-4b84-b507-b5bb834fd19e admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:11.533568 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36942049-7382-47cc-b115-f4642febda57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:11.534221 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36942049-7382-47cc-b115-f4642febda57 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:11.534497 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36942049-7382-47cc-b115-f4642febda57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:11.534779 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36942049-7382-47cc-b115-f4642febda57 admin admin] PATH_INFO: `/v3/projects/8063874d575a4618bcab79608b3970e0/users/4ee20fe29c964bcbbfa764fd469050ba/roles/74a40bc819cb42eea7e966a4b2a016f1` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:11.551729 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36942049-7382-47cc-b115-f4642febda57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8063874d575a4618bcab79608b3970e0, user_id=4ee20fe29c964bcbbfa764fd469050ba, role_id=74a40bc819cb42eea7e966a4b2a016f1)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:11.554465 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36942049-7382-47cc-b115-f4642febda57 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:11.568839 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 934/1875] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:19:11 2022] PUT /identity/v3/projects/8063874d575a4618bcab79608b3970e0/users/4ee20fe29c964bcbbfa764fd469050ba/roles/74a40bc819cb42eea7e966a4b2a016f1 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 19:19:11.581208 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8dd7a9-234c-4953-aab1-43991c54f8a9 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:11.581492 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8dd7a9-234c-4953-aab1-43991c54f8a9 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:11.581764 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8dd7a9-234c-4953-aab1-43991c54f8a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:11.615118 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-6e8dd7a9-234c-4953-aab1-43991c54f8a9 None None] MFA Rules not processed for user `4ee20fe29c964bcbbfa764fd469050ba`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:19:11.616735 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-6e8dd7a9-234c-4953-aab1-43991c54f8a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:11.693119 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 942/1876] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:19:11 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:19:11.699835 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36942049-7382-47cc-b115-f4642febda57 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:11.741923 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8dd7a9-234c-4953-aab1-43991c54f8a9 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:11.743408 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-6e8dd7a9-234c-4953-aab1-43991c54f8a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:11.747037 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:19:11.752096 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:19:11.802879 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36942049-7382-47cc-b115-f4642febda57 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:11.809540 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6107dab-78ee-488e-842e-cfcc7a1100d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:11.810359 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6107dab-78ee-488e-842e-cfcc7a1100d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:11.810875 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6107dab-78ee-488e-842e-cfcc7a1100d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:11.811291 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6107dab-78ee-488e-842e-cfcc7a1100d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:11.812176 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6107dab-78ee-488e-842e-cfcc7a1100d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:11.814597 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6107dab-78ee-488e-842e-cfcc7a1100d8 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:11.855779 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 935/1877] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 19:19:11 2022] POST /identity/v3/projects => generated 377 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:19:11.860160 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6107dab-78ee-488e-842e-cfcc7a1100d8 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:11.869302 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8dd7a9-234c-4953-aab1-43991c54f8a9 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:11.939290 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6107dab-78ee-488e-842e-cfcc7a1100d8 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:11.944057 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05539610-fadd-46d5-8d68-269b03d5a6fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:11.944742 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05539610-fadd-46d5-8d68-269b03d5a6fc admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:11.945087 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05539610-fadd-46d5-8d68-269b03d5a6fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:11.945425 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05539610-fadd-46d5-8d68-269b03d5a6fc admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:11.946129 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05539610-fadd-46d5-8d68-269b03d5a6fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:11.948357 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05539610-fadd-46d5-8d68-269b03d5a6fc admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:11.949731 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8facaa11-3db2-4e35-a08f-08d952d2c58c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:11.950454 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8facaa11-3db2-4e35-a08f-08d952d2c58c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:11.950799 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8facaa11-3db2-4e35-a08f-08d952d2c58c admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:11.951240 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8facaa11-3db2-4e35-a08f-08d952d2c58c admin admin] PATH_INFO: `/v3/projects/d986814d4fc8472ea26b4e2163c91989` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:11.952677 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8facaa11-3db2-4e35-a08f-08d952d2c58c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d986814d4fc8472ea26b4e2163c91989)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:11.955085 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8facaa11-3db2-4e35-a08f-08d952d2c58c admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:11.982163 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 936/1878] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 19:19:11 2022] POST /identity/v3/users => generated 289 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:19:11.984964 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05539610-fadd-46d5-8d68-269b03d5a6fc admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:12.011948 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05539610-fadd-46d5-8d68-269b03d5a6fc admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:12.018822 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a40d0ce-a1a7-4daf-9862-2ded33e3dfd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:12.018822 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a40d0ce-a1a7-4daf-9862-2ded33e3dfd8 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:12.018822 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a40d0ce-a1a7-4daf-9862-2ded33e3dfd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:12.018822 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a40d0ce-a1a7-4daf-9862-2ded33e3dfd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:12.019908 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a40d0ce-a1a7-4daf-9862-2ded33e3dfd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:12.021937 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a40d0ce-a1a7-4daf-9862-2ded33e3dfd8 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:12.041654 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 937/1879] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:19:11 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 19:19:12.046401 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a40d0ce-a1a7-4daf-9862-2ded33e3dfd8 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:12.050469 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-8facaa11-3db2-4e35-a08f-08d952d2c58c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd986814d4fc8472ea26b4e2163c91989', 'request_id': 'req-8facaa11-3db2-4e35-a08f-08d952d2c58c'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:12.060125 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a40d0ce-a1a7-4daf-9862-2ded33e3dfd8 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:12.068323 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-8facaa11-3db2-4e35-a08f-08d952d2c58c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd986814d4fc8472ea26b4e2163c91989', 'request_id': 'req-8facaa11-3db2-4e35-a08f-08d952d2c58c'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:12.079946 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-8facaa11-3db2-4e35-a08f-08d952d2c58c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd986814d4fc8472ea26b4e2163c91989', 'request_id': 'req-8facaa11-3db2-4e35-a08f-08d952d2c58c'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:12.113820 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-8facaa11-3db2-4e35-a08f-08d952d2c58c admin admin] The token cache is being invalidate because project d986814d4fc8472ea26b4e2163c91989 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 19:19:12.114538 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-8facaa11-3db2-4e35-a08f-08d952d2c58c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:12.115322 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-8facaa11-3db2-4e35-a08f-08d952d2c58c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:12.117802 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 943/1880] 10.222.0.61 () {58 vars in 1233 bytes} [Mon Aug 22 19:19:11 2022] DELETE /identity/v3/projects/d986814d4fc8472ea26b4e2163c91989 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 19:19:12.122639 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbd8dca-5a59-45f4-b38a-c0deb1381292 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:12.122948 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbd8dca-5a59-45f4-b38a-c0deb1381292 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:12.123432 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbd8dca-5a59-45f4-b38a-c0deb1381292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:12.145337 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-6dbd8dca-5a59-45f4-b38a-c0deb1381292 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:19:12.148516 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-6dbd8dca-5a59-45f4-b38a-c0deb1381292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:12.160546 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564aad0d-d30f-4893-96af-26ee5fd857e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:12.161160 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564aad0d-d30f-4893-96af-26ee5fd857e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:12.161436 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564aad0d-d30f-4893-96af-26ee5fd857e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:12.161708 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564aad0d-d30f-4893-96af-26ee5fd857e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:12.162374 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564aad0d-d30f-4893-96af-26ee5fd857e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:12.163990 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564aad0d-d30f-4893-96af-26ee5fd857e1 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:12.176024 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 938/1881] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:19:12 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 19:19:12.180186 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564aad0d-d30f-4893-96af-26ee5fd857e1 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:12.181973 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-564aad0d-d30f-4893-96af-26ee5fd857e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:12.195164 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:19:12.197586 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564aad0d-d30f-4893-96af-26ee5fd857e1 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:12.204097 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6f15e7-627c-4bd0-9c7b-ad967f652798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:12.205054 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6f15e7-627c-4bd0-9c7b-ad967f652798 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:12.205480 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6f15e7-627c-4bd0-9c7b-ad967f652798 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:12.206109 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6f15e7-627c-4bd0-9c7b-ad967f652798 admin admin] PATH_INFO: `/v3/projects/fe9abbaf91794fa69709b1bf0c06d553/users/639c7142edb34972add6a9d453be5582/roles/74a40bc819cb42eea7e966a4b2a016f1` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:12.222518 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 944/1882] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:19:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 19:19:12.223283 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6f15e7-627c-4bd0-9c7b-ad967f652798 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe9abbaf91794fa69709b1bf0c06d553, user_id=639c7142edb34972add6a9d453be5582, role_id=74a40bc819cb42eea7e966a4b2a016f1)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:12.225617 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6f15e7-627c-4bd0-9c7b-ad967f652798 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:12.230379 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633c155b-dd6d-421d-afd6-4445ac8b1114 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:12.230668 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633c155b-dd6d-421d-afd6-4445ac8b1114 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:12.230992 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633c155b-dd6d-421d-afd6-4445ac8b1114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:12.240327 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 939/1883] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:19:12 2022] PUT /identity/v3/projects/fe9abbaf91794fa69709b1bf0c06d553/users/639c7142edb34972add6a9d453be5582/roles/74a40bc819cb42eea7e966a4b2a016f1 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:19:12.254266 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-633c155b-dd6d-421d-afd6-4445ac8b1114 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:19:12.255592 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-633c155b-dd6d-421d-afd6-4445ac8b1114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:12.256732 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7123ef1-15a3-4366-9de7-90ac3fe2e15e None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:12.256969 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7123ef1-15a3-4366-9de7-90ac3fe2e15e None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:12.257255 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7123ef1-15a3-4366-9de7-90ac3fe2e15e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:12.273318 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:19:12.290164 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-b7123ef1-15a3-4366-9de7-90ac3fe2e15e None None] MFA Rules not processed for user `639c7142edb34972add6a9d453be5582`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:19:12.291413 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-b7123ef1-15a3-4366-9de7-90ac3fe2e15e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:12.320427 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 945/1884] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:19:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 19:19:12.325054 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633c155b-dd6d-421d-afd6-4445ac8b1114 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:12.333773 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633c155b-dd6d-421d-afd6-4445ac8b1114 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:12.338623 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941ac5ec-af17-4f9b-b8bd-6c0a59624f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:12.339998 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941ac5ec-af17-4f9b-b8bd-6c0a59624f30 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:12.340306 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941ac5ec-af17-4f9b-b8bd-6c0a59624f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:12.340580 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941ac5ec-af17-4f9b-b8bd-6c0a59624f30 admin admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:12.341232 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941ac5ec-af17-4f9b-b8bd-6c0a59624f30 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:19:12.341533 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941ac5ec-af17-4f9b-b8bd-6c0a59624f30 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:12.343455 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941ac5ec-af17-4f9b-b8bd-6c0a59624f30 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:12.355553 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 946/1885] 10.222.0.61 () {58 vars in 1153 bytes} [Mon Aug 22 19:19:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 19:19:12.359188 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 940/1886] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:19:12 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:19:12.359849 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941ac5ec-af17-4f9b-b8bd-6c0a59624f30 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:12.362202 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7123ef1-15a3-4366-9de7-90ac3fe2e15e None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:12.369513 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941ac5ec-af17-4f9b-b8bd-6c0a59624f30 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:12.372861 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7123ef1-15a3-4366-9de7-90ac3fe2e15e None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:12.374182 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ac82ca-6313-4350-8b57-6fff577880be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:12.374769 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ac82ca-6313-4350-8b57-6fff577880be admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:12.375043 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ac82ca-6313-4350-8b57-6fff577880be admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:12.375314 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ac82ca-6313-4350-8b57-6fff577880be admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:12.375978 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ac82ca-6313-4350-8b57-6fff577880be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:12.377461 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd209c1e-bdcc-4ff3-a5da-69ec2e452b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:12.377839 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ac82ca-6313-4350-8b57-6fff577880be admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:12.378075 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd209c1e-bdcc-4ff3-a5da-69ec2e452b3d admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:12.378352 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd209c1e-bdcc-4ff3-a5da-69ec2e452b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:12.378622 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd209c1e-bdcc-4ff3-a5da-69ec2e452b3d admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:12.379265 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd209c1e-bdcc-4ff3-a5da-69ec2e452b3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:12.381313 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd209c1e-bdcc-4ff3-a5da-69ec2e452b3d admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:12.409572 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 947/1887] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 19:19:12 2022] POST /identity/v3/projects => generated 407 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 19:19:12.412684 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 941/1888] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 19:19:12 2022] POST /identity/v3/projects => generated 377 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 19:19:12.413034 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ac82ca-6313-4350-8b57-6fff577880be admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:12.413971 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd209c1e-bdcc-4ff3-a5da-69ec2e452b3d admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:12.481578 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ac82ca-6313-4350-8b57-6fff577880be admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:12.481837 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd209c1e-bdcc-4ff3-a5da-69ec2e452b3d admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:12.486652 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56022f4a-e7bd-4080-8414-ab8360e44b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:12.486792 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fff7e49-3189-4e89-8354-1a9659ab3606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:12.487262 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fff7e49-3189-4e89-8354-1a9659ab3606 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:12.487388 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56022f4a-e7bd-4080-8414-ab8360e44b2a admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:12.487536 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fff7e49-3189-4e89-8354-1a9659ab3606 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:12.487619 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56022f4a-e7bd-4080-8414-ab8360e44b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:12.487845 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fff7e49-3189-4e89-8354-1a9659ab3606 admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:12.487953 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56022f4a-e7bd-4080-8414-ab8360e44b2a admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:12.488471 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fff7e49-3189-4e89-8354-1a9659ab3606 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:12.488608 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56022f4a-e7bd-4080-8414-ab8360e44b2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:12.490388 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fff7e49-3189-4e89-8354-1a9659ab3606 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:12.490558 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56022f4a-e7bd-4080-8414-ab8360e44b2a admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:12.516254 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 942/1889] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 19:19:12 2022] POST /identity/v3/users => generated 295 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:19:12.520166 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56022f4a-e7bd-4080-8414-ab8360e44b2a admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:12.524870 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 948/1890] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 19:19:12 2022] POST /identity/v3/users => generated 304 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:19:12.525729 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fff7e49-3189-4e89-8354-1a9659ab3606 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:12.532745 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56022f4a-e7bd-4080-8414-ab8360e44b2a admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:12.536352 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fff7e49-3189-4e89-8354-1a9659ab3606 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:12.537436 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0d67b3-d86b-4fa2-ab2c-db2abb6bbb2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:12.538300 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0d67b3-d86b-4fa2-ab2c-db2abb6bbb2f admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:12.538686 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0d67b3-d86b-4fa2-ab2c-db2abb6bbb2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:12.539052 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0d67b3-d86b-4fa2-ab2c-db2abb6bbb2f admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:12.540041 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0d67b3-d86b-4fa2-ab2c-db2abb6bbb2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:12.540967 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914ccb4d-189f-4628-9088-74f477ea4a48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:12.541572 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914ccb4d-189f-4628-9088-74f477ea4a48 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:12.541864 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914ccb4d-189f-4628-9088-74f477ea4a48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:12.542114 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0d67b3-d86b-4fa2-ab2c-db2abb6bbb2f admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:12.542209 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914ccb4d-189f-4628-9088-74f477ea4a48 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:12.542795 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914ccb4d-189f-4628-9088-74f477ea4a48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:12.544534 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914ccb4d-189f-4628-9088-74f477ea4a48 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:12.554809 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 943/1891] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:19:12 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 19:19:12.555812 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 949/1892] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:19:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:19:12.560636 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0d67b3-d86b-4fa2-ab2c-db2abb6bbb2f admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:12.562326 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914ccb4d-189f-4628-9088-74f477ea4a48 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:12.572136 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0d67b3-d86b-4fa2-ab2c-db2abb6bbb2f admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:12.573784 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914ccb4d-189f-4628-9088-74f477ea4a48 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:12.578514 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19665e1-d816-41c7-8b2a-d8ca7f14c5d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:12.579195 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19665e1-d816-41c7-8b2a-d8ca7f14c5d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:12.579547 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19665e1-d816-41c7-8b2a-d8ca7f14c5d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:12.579915 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19665e1-d816-41c7-8b2a-d8ca7f14c5d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:12.580171 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54f85f0-d5f7-4586-99b1-538b39397ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:12.580703 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19665e1-d816-41c7-8b2a-d8ca7f14c5d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:12.581169 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54f85f0-d5f7-4586-99b1-538b39397ec4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:12.581588 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54f85f0-d5f7-4586-99b1-538b39397ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:12.582001 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54f85f0-d5f7-4586-99b1-538b39397ec4 admin admin] PATH_INFO: `/v3/projects/07088db66a404b62ba31975d711df4bf/users/5c83bd4a0576403dba33b273cbeb66ed/roles/163c30c3b3c246beb849ce176f02d9da` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:12.582561 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19665e1-d816-41c7-8b2a-d8ca7f14c5d6 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:12.593732 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 950/1893] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:19:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 19:19:12.597732 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54f85f0-d5f7-4586-99b1-538b39397ec4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07088db66a404b62ba31975d711df4bf, user_id=5c83bd4a0576403dba33b273cbeb66ed, role_id=163c30c3b3c246beb849ce176f02d9da)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:12.597870 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19665e1-d816-41c7-8b2a-d8ca7f14c5d6 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:12.600546 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54f85f0-d5f7-4586-99b1-538b39397ec4 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:12.607837 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19665e1-d816-41c7-8b2a-d8ca7f14c5d6 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:12.613981 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 944/1894] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:19:12 2022] PUT /identity/v3/projects/07088db66a404b62ba31975d711df4bf/users/5c83bd4a0576403dba33b273cbeb66ed/roles/163c30c3b3c246beb849ce176f02d9da => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:19:12.617932 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54f85f0-d5f7-4586-99b1-538b39397ec4 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:12.619470 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:19:12.651221 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:19:12.667113 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeba19d7-c2ed-4ad7-8861-f1b62c89ac21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:12.667926 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeba19d7-c2ed-4ad7-8861-f1b62c89ac21 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:12.668233 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeba19d7-c2ed-4ad7-8861-f1b62c89ac21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:12.668557 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeba19d7-c2ed-4ad7-8861-f1b62c89ac21 admin admin] PATH_INFO: `/v3/projects/371250516b4143ceb2fd1bbb4eec6e1c/users/bec0448f69ef4948b2190a2cff97bb04/roles/74a40bc819cb42eea7e966a4b2a016f1` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:12.683147 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeba19d7-c2ed-4ad7-8861-f1b62c89ac21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=371250516b4143ceb2fd1bbb4eec6e1c, user_id=bec0448f69ef4948b2190a2cff97bb04, role_id=74a40bc819cb42eea7e966a4b2a016f1)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:12.683812 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54f85f0-d5f7-4586-99b1-538b39397ec4 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:12.685836 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeba19d7-c2ed-4ad7-8861-f1b62c89ac21 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:12.688706 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108d601c-ba3f-44e7-9e9a-980f181e1dd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:12.689319 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108d601c-ba3f-44e7-9e9a-980f181e1dd6 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:12.689597 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108d601c-ba3f-44e7-9e9a-980f181e1dd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:12.689865 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108d601c-ba3f-44e7-9e9a-980f181e1dd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:12.690640 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108d601c-ba3f-44e7-9e9a-980f181e1dd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:12.692434 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108d601c-ba3f-44e7-9e9a-980f181e1dd6 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:12.699375 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 951/1895] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:19:12 2022] PUT /identity/v3/projects/371250516b4143ceb2fd1bbb4eec6e1c/users/bec0448f69ef4948b2190a2cff97bb04/roles/74a40bc819cb42eea7e966a4b2a016f1 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 19:19:12.703197 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 945/1896] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:19:12 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:19:12.707054 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeba19d7-c2ed-4ad7-8861-f1b62c89ac21 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:12.710582 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7785a7c3-172a-4110-b22b-a4eb0d90ab1d None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:12.710870 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7785a7c3-172a-4110-b22b-a4eb0d90ab1d None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:12.711138 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7785a7c3-172a-4110-b22b-a4eb0d90ab1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:12.738806 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-7785a7c3-172a-4110-b22b-a4eb0d90ab1d None None] MFA Rules not processed for user `bec0448f69ef4948b2190a2cff97bb04`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:19:12.740048 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-7785a7c3-172a-4110-b22b-a4eb0d90ab1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:12.769322 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeba19d7-c2ed-4ad7-8861-f1b62c89ac21 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:12.774108 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef616b5c-e5a6-4b9a-afa8-2ff10c4062e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:12.774773 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef616b5c-e5a6-4b9a-afa8-2ff10c4062e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:12.775042 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef616b5c-e5a6-4b9a-afa8-2ff10c4062e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:12.775309 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef616b5c-e5a6-4b9a-afa8-2ff10c4062e1 admin admin] PATH_INFO: `/v3/projects/07088db66a404b62ba31975d711df4bf/users/5c83bd4a0576403dba33b273cbeb66ed/roles/163c30c3b3c246beb849ce176f02d9da` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:12.777274 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef616b5c-e5a6-4b9a-afa8-2ff10c4062e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07088db66a404b62ba31975d711df4bf, user_id=5c83bd4a0576403dba33b273cbeb66ed, role_id=163c30c3b3c246beb849ce176f02d9da)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:12.779986 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef616b5c-e5a6-4b9a-afa8-2ff10c4062e1 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:12.790090 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 952/1897] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:19:12 2022] PUT /identity/v3/projects/07088db66a404b62ba31975d711df4bf/users/5c83bd4a0576403dba33b273cbeb66ed/roles/163c30c3b3c246beb849ce176f02d9da => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:19:12.800619 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97726a0b-7156-4f54-af6c-5b1e43aea809 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:12.800903 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97726a0b-7156-4f54-af6c-5b1e43aea809 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:12.801175 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97726a0b-7156-4f54-af6c-5b1e43aea809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:12.806306 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 946/1898] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:19:12 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 19:19:12.812337 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7785a7c3-172a-4110-b22b-a4eb0d90ab1d None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:12.824549 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7785a7c3-172a-4110-b22b-a4eb0d90ab1d None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:12.834943 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-97726a0b-7156-4f54-af6c-5b1e43aea809 None None] MFA Rules not processed for user `5c83bd4a0576403dba33b273cbeb66ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:19:12.836468 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-97726a0b-7156-4f54-af6c-5b1e43aea809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:12.920569 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a47645-ed94-4b01-a4e8-e83f005dcabf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:12.922039 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a47645-ed94-4b01-a4e8-e83f005dcabf admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:12.924587 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a47645-ed94-4b01-a4e8-e83f005dcabf admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:12.924587 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a47645-ed94-4b01-a4e8-e83f005dcabf admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:12.924842 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a47645-ed94-4b01-a4e8-e83f005dcabf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:12.926679 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a47645-ed94-4b01-a4e8-e83f005dcabf admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:12.945000 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 953/1899] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:19:12 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 19:19:12.958840 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97726a0b-7156-4f54-af6c-5b1e43aea809 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:12.960926 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-97726a0b-7156-4f54-af6c-5b1e43aea809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:12.975950 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 947/1900] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 19:19:12 2022] POST /identity/v3/projects => generated 409 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 19:19:12.978814 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a47645-ed94-4b01-a4e8-e83f005dcabf admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:12.984970 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:19:13.027150 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97726a0b-7156-4f54-af6c-5b1e43aea809 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:13.034366 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:19:13.061099 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a47645-ed94-4b01-a4e8-e83f005dcabf admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:13.065883 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb53438d-c4b0-45ea-877a-477d4204edbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:13.066505 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb53438d-c4b0-45ea-877a-477d4204edbc admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:13.066783 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb53438d-c4b0-45ea-877a-477d4204edbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:13.067049 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb53438d-c4b0-45ea-877a-477d4204edbc admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:13.067709 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb53438d-c4b0-45ea-877a-477d4204edbc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:13.070362 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb53438d-c4b0-45ea-877a-477d4204edbc admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:13.079100 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69f4523-d728-431d-b874-818c71c14edc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e79498873d4ed29a0e19bbea9b9214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:13.079686 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69f4523-d728-431d-b874-818c71c14edc service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:13.080038 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69f4523-d728-431d-b874-818c71c14edc service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:13.080308 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69f4523-d728-431d-b874-818c71c14edc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:13.095635 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 948/1901] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 19:19:12 2022] POST /identity/v3/users => generated 311 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:19:13.099248 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb53438d-c4b0-45ea-877a-477d4204edbc admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:13.109716 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb53438d-c4b0-45ea-877a-477d4204edbc admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:13.121917 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e03ac3-1345-44ad-b4b9-e25f76a11fda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:13.123789 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e03ac3-1345-44ad-b4b9-e25f76a11fda admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:13.124145 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e03ac3-1345-44ad-b4b9-e25f76a11fda admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:13.124486 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e03ac3-1345-44ad-b4b9-e25f76a11fda admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:13.125208 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e03ac3-1345-44ad-b4b9-e25f76a11fda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:13.127068 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e03ac3-1345-44ad-b4b9-e25f76a11fda admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:13.152975 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 949/1902] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:19:13 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 19:19:13.153900 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e03ac3-1345-44ad-b4b9-e25f76a11fda admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:13.163989 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69f4523-d728-431d-b874-818c71c14edc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:13.165800 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69f4523-d728-431d-b874-818c71c14edc service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:13.168487 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e03ac3-1345-44ad-b4b9-e25f76a11fda admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:13.176051 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d8347e-f5ec-4f00-9240-fd07d95716dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:13.176785 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d8347e-f5ec-4f00-9240-fd07d95716dc admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:13.177010 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 954/1903] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:19:12 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:19:13.177243 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d8347e-f5ec-4f00-9240-fd07d95716dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:13.177584 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d8347e-f5ec-4f00-9240-fd07d95716dc admin admin] PATH_INFO: `/v3/projects/0a506d8d74844f9fb369ed96977e68e4/users/8aaea49abc3e408495ba38aaee34e70b/roles/163c30c3b3c246beb849ce176f02d9da` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:13.198828 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d8347e-f5ec-4f00-9240-fd07d95716dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a506d8d74844f9fb369ed96977e68e4, user_id=8aaea49abc3e408495ba38aaee34e70b, role_id=163c30c3b3c246beb849ce176f02d9da)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:13.201739 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d8347e-f5ec-4f00-9240-fd07d95716dc admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:13.215790 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 950/1904] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:19:13 2022] PUT /identity/v3/projects/0a506d8d74844f9fb369ed96977e68e4/users/8aaea49abc3e408495ba38aaee34e70b/roles/163c30c3b3c246beb849ce176f02d9da => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 19:19:13.219965 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69f4523-d728-431d-b874-818c71c14edc service cinder] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:13.224559 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d8347e-f5ec-4f00-9240-fd07d95716dc admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:13.226399 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-45d8347e-f5ec-4f00-9240-fd07d95716dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:13.231721 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69f4523-d728-431d-b874-818c71c14edc service cinder] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:13.289326 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d8347e-f5ec-4f00-9240-fd07d95716dc admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:13.294354 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fe5a41-0430-4485-b6b2-76a5b9a45baa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e79498873d4ed29a0e19bbea9b9214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:13.295330 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fe5a41-0430-4485-b6b2-76a5b9a45baa service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:13.295598 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fe5a41-0430-4485-b6b2-76a5b9a45baa service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:13.295904 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fe5a41-0430-4485-b6b2-76a5b9a45baa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:13.298060 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-07fe5a41-0430-4485-b6b2-76a5b9a45baa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:13.310980 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa640cd-5625-455f-b7f3-d2be5fe763d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:13.311954 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa640cd-5625-455f-b7f3-d2be5fe763d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:13.312331 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa640cd-5625-455f-b7f3-d2be5fe763d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:13.319793 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa640cd-5625-455f-b7f3-d2be5fe763d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:13.320889 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa640cd-5625-455f-b7f3-d2be5fe763d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:13.322658 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa640cd-5625-455f-b7f3-d2be5fe763d6 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:13.340165 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 955/1905] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:19:13 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:19:13.343798 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa640cd-5625-455f-b7f3-d2be5fe763d6 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:13.359388 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa640cd-5625-455f-b7f3-d2be5fe763d6 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:13.365532 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b5e0d3-631c-40da-83f5-d10f82426a4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:13.366347 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b5e0d3-631c-40da-83f5-d10f82426a4b admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:13.366667 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b5e0d3-631c-40da-83f5-d10f82426a4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:13.367113 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b5e0d3-631c-40da-83f5-d10f82426a4b admin admin] PATH_INFO: `/v3/projects/0a506d8d74844f9fb369ed96977e68e4/users/8aaea49abc3e408495ba38aaee34e70b/roles/163c30c3b3c246beb849ce176f02d9da` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:13.371611 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b5e0d3-631c-40da-83f5-d10f82426a4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a506d8d74844f9fb369ed96977e68e4, user_id=8aaea49abc3e408495ba38aaee34e70b, role_id=163c30c3b3c246beb849ce176f02d9da)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:13.374304 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b5e0d3-631c-40da-83f5-d10f82426a4b admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:13.375377 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fe5a41-0430-4485-b6b2-76a5b9a45baa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:13.376977 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fe5a41-0430-4485-b6b2-76a5b9a45baa service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:13.386545 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 956/1906] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:19:13 2022] PUT /identity/v3/projects/0a506d8d74844f9fb369ed96977e68e4/users/8aaea49abc3e408495ba38aaee34e70b/roles/163c30c3b3c246beb849ce176f02d9da => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 19:19:13.393064 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 951/1907] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:19:13 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 19:19:13.396538 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce892b3-87a9-4f01-b1a1-cd13a2269a67 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:13.396834 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce892b3-87a9-4f01-b1a1-cd13a2269a67 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:13.397114 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce892b3-87a9-4f01-b1a1-cd13a2269a67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:13.411261 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:19:13.430282 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-dce892b3-87a9-4f01-b1a1-cd13a2269a67 None None] MFA Rules not processed for user `8aaea49abc3e408495ba38aaee34e70b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:19:13.430989 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-dce892b3-87a9-4f01-b1a1-cd13a2269a67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:13.508977 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 957/1908] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:19:13 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:19:13.520754 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fe5a41-0430-4485-b6b2-76a5b9a45baa service cinder] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:13.529341 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fe5a41-0430-4485-b6b2-76a5b9a45baa service cinder] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:13.579508 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:19:13.586302 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29e4075-2076-4284-8756-1a33633fca25 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e79498873d4ed29a0e19bbea9b9214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:13.587043 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29e4075-2076-4284-8756-1a33633fca25 service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:13.587674 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29e4075-2076-4284-8756-1a33633fca25 service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:13.587804 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29e4075-2076-4284-8756-1a33633fca25 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:13.598282 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29e4075-2076-4284-8756-1a33633fca25 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:13.599800 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29e4075-2076-4284-8756-1a33633fca25 service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:13.610731 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 952/1909] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:19:13 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:19:13.821180 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce892b3-87a9-4f01-b1a1-cd13a2269a67 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:13.831025 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce892b3-87a9-4f01-b1a1-cd13a2269a67 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:13.834882 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0655f4-734e-49b7-b1cd-eb2b65f389b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e79498873d4ed29a0e19bbea9b9214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:13.835515 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0655f4-734e-49b7-b1cd-eb2b65f389b4 service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:13.835827 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0655f4-734e-49b7-b1cd-eb2b65f389b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:13.836096 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0655f4-734e-49b7-b1cd-eb2b65f389b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:13.892102 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0655f4-734e-49b7-b1cd-eb2b65f389b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:13.893458 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0655f4-734e-49b7-b1cd-eb2b65f389b4 service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:13.906784 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 958/1910] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:19:13 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:19:23.606105 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=4 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:19:23.898267 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=59 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:19:35.407011 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29e4075-2076-4284-8756-1a33633fca25 service cinder] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:35.470911 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29e4075-2076-4284-8756-1a33633fca25 service cinder] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:35.475957 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cb30aa-9f39-47f2-b6ef-dc27395a451e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:35.476440 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cb30aa-9f39-47f2-b6ef-dc27395a451e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:35.476980 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cb30aa-9f39-47f2-b6ef-dc27395a451e admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:35.477420 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cb30aa-9f39-47f2-b6ef-dc27395a451e admin admin] PATH_INFO: `/v3/users/4ee20fe29c964bcbbfa764fd469050ba` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:35.478487 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cb30aa-9f39-47f2-b6ef-dc27395a451e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ee20fe29c964bcbbfa764fd469050ba)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:35.480959 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cb30aa-9f39-47f2-b6ef-dc27395a451e admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:35.553134 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-33cb30aa-9f39-47f2-b6ef-dc27395a451e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ee20fe29c964bcbbfa764fd469050ba'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:35.554121 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-33cb30aa-9f39-47f2-b6ef-dc27395a451e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ee20fe29c964bcbbfa764fd469050ba'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:35.565494 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-33cb30aa-9f39-47f2-b6ef-dc27395a451e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ee20fe29c964bcbbfa764fd469050ba'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:35.575202 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-33cb30aa-9f39-47f2-b6ef-dc27395a451e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ee20fe29c964bcbbfa764fd469050ba'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:35.596809 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-33cb30aa-9f39-47f2-b6ef-dc27395a451e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ee20fe29c964bcbbfa764fd469050ba'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:35.609499 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 953/1911] 10.222.0.61 () {58 vars in 1224 bytes} [Mon Aug 22 19:19:35 2022] DELETE /identity/v3/users/4ee20fe29c964bcbbfa764fd469050ba => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:19:35.611211 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bbfd0d-d808-41ba-98f9-1670d70e3611 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:35.612418 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bbfd0d-d808-41ba-98f9-1670d70e3611 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:35.612899 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bbfd0d-d808-41ba-98f9-1670d70e3611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:35.638509 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-57bbfd0d-d808-41ba-98f9-1670d70e3611 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:19:35.639857 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-57bbfd0d-d808-41ba-98f9-1670d70e3611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:35.731260 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 959/1912] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:19:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:19:35.741295 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cb30aa-9f39-47f2-b6ef-dc27395a451e admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:35.742922 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-33cb30aa-9f39-47f2-b6ef-dc27395a451e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:35.834953 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cb30aa-9f39-47f2-b6ef-dc27395a451e admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:35.844543 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7635b776-be24-41f2-a023-0cfbc85df0f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:35.845189 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7635b776-be24-41f2-a023-0cfbc85df0f0 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:35.845467 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7635b776-be24-41f2-a023-0cfbc85df0f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:35.845741 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7635b776-be24-41f2-a023-0cfbc85df0f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:35.874116 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7635b776-be24-41f2-a023-0cfbc85df0f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:35.878824 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7635b776-be24-41f2-a023-0cfbc85df0f0 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:35.905320 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 954/1913] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:19:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:19:36.286952 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bbfd0d-d808-41ba-98f9-1670d70e3611 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:36.290055 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-57bbfd0d-d808-41ba-98f9-1670d70e3611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:36.314734 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bbfd0d-d808-41ba-98f9-1670d70e3611 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:36.320642 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66e0259-9b12-4aa8-8e9f-8ae4b220d0cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:36.321405 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66e0259-9b12-4aa8-8e9f-8ae4b220d0cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:36.321747 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66e0259-9b12-4aa8-8e9f-8ae4b220d0cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:36.322084 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66e0259-9b12-4aa8-8e9f-8ae4b220d0cd admin admin] PATH_INFO: `/v3/projects/8063874d575a4618bcab79608b3970e0` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:36.323929 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66e0259-9b12-4aa8-8e9f-8ae4b220d0cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=8063874d575a4618bcab79608b3970e0)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:36.326005 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66e0259-9b12-4aa8-8e9f-8ae4b220d0cd admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:36.377696 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-d66e0259-9b12-4aa8-8e9f-8ae4b220d0cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8063874d575a4618bcab79608b3970e0', 'request_id': 'req-d66e0259-9b12-4aa8-8e9f-8ae4b220d0cd'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:36.387821 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-d66e0259-9b12-4aa8-8e9f-8ae4b220d0cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8063874d575a4618bcab79608b3970e0', 'request_id': 'req-d66e0259-9b12-4aa8-8e9f-8ae4b220d0cd'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:36.394557 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-d66e0259-9b12-4aa8-8e9f-8ae4b220d0cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8063874d575a4618bcab79608b3970e0', 'request_id': 'req-d66e0259-9b12-4aa8-8e9f-8ae4b220d0cd'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:36.406813 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-d66e0259-9b12-4aa8-8e9f-8ae4b220d0cd admin admin] The token cache is being invalidate because project 8063874d575a4618bcab79608b3970e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 19:19:36.407110 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-d66e0259-9b12-4aa8-8e9f-8ae4b220d0cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:36.408345 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-d66e0259-9b12-4aa8-8e9f-8ae4b220d0cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:36.412784 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 960/1914] 10.222.0.61 () {58 vars in 1233 bytes} [Mon Aug 22 19:19:36 2022] DELETE /identity/v3/projects/8063874d575a4618bcab79608b3970e0 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:19:36.414559 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7635b776-be24-41f2-a023-0cfbc85df0f0 service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:36.425648 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7635b776-be24-41f2-a023-0cfbc85df0f0 service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:36.454945 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:19:36.503893 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fc2e6f-6ee1-4a73-b732-d3f9d5fa33ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:36.504601 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fc2e6f-6ee1-4a73-b732-d3f9d5fa33ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:36.504890 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fc2e6f-6ee1-4a73-b732-d3f9d5fa33ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:36.511692 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fc2e6f-6ee1-4a73-b732-d3f9d5fa33ec admin admin] PATH_INFO: `/v3/users/639c7142edb34972add6a9d453be5582` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:36.511692 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fc2e6f-6ee1-4a73-b732-d3f9d5fa33ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=639c7142edb34972add6a9d453be5582)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:36.511692 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fc2e6f-6ee1-4a73-b732-d3f9d5fa33ec admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:36.568375 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-e5fc2e6f-6ee1-4a73-b732-d3f9d5fa33ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '639c7142edb34972add6a9d453be5582'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:36.569067 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-e5fc2e6f-6ee1-4a73-b732-d3f9d5fa33ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '639c7142edb34972add6a9d453be5582'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:36.577527 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-e5fc2e6f-6ee1-4a73-b732-d3f9d5fa33ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '639c7142edb34972add6a9d453be5582'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:36.586173 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-e5fc2e6f-6ee1-4a73-b732-d3f9d5fa33ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '639c7142edb34972add6a9d453be5582'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:36.599400 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-e5fc2e6f-6ee1-4a73-b732-d3f9d5fa33ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '639c7142edb34972add6a9d453be5582'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:36.606201 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 955/1915] 10.222.0.61 () {58 vars in 1224 bytes} [Mon Aug 22 19:19:36 2022] DELETE /identity/v3/users/639c7142edb34972add6a9d453be5582 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:19:36.855007 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66e0259-9b12-4aa8-8e9f-8ae4b220d0cd admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:36.856613 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-d66e0259-9b12-4aa8-8e9f-8ae4b220d0cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:36.870286 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66e0259-9b12-4aa8-8e9f-8ae4b220d0cd admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:36.958831 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16893404-47d2-47dc-b59c-8cb9bfdcc756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:36.959745 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16893404-47d2-47dc-b59c-8cb9bfdcc756 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:36.960177 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16893404-47d2-47dc-b59c-8cb9bfdcc756 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:36.960676 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16893404-47d2-47dc-b59c-8cb9bfdcc756 admin admin] PATH_INFO: `/v3/users/bec0448f69ef4948b2190a2cff97bb04` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:36.962171 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16893404-47d2-47dc-b59c-8cb9bfdcc756 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bec0448f69ef4948b2190a2cff97bb04)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:36.965207 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16893404-47d2-47dc-b59c-8cb9bfdcc756 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:37.002538 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fc2e6f-6ee1-4a73-b732-d3f9d5fa33ec admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:37.004063 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-e5fc2e6f-6ee1-4a73-b732-d3f9d5fa33ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:37.033758 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fc2e6f-6ee1-4a73-b732-d3f9d5fa33ec admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:37.037794 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:19:37.040350 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69366071-b693-47b2-83ab-18fed160e631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:37.041149 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69366071-b693-47b2-83ab-18fed160e631 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:37.042177 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69366071-b693-47b2-83ab-18fed160e631 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:37.042508 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69366071-b693-47b2-83ab-18fed160e631 admin admin] PATH_INFO: `/v3/projects/fe9abbaf91794fa69709b1bf0c06d553` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:37.044503 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69366071-b693-47b2-83ab-18fed160e631 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe9abbaf91794fa69709b1bf0c06d553)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:37.046520 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69366071-b693-47b2-83ab-18fed160e631 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:37.073185 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-16893404-47d2-47dc-b59c-8cb9bfdcc756 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bec0448f69ef4948b2190a2cff97bb04'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:37.073869 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-16893404-47d2-47dc-b59c-8cb9bfdcc756 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bec0448f69ef4948b2190a2cff97bb04'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:37.085018 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-16893404-47d2-47dc-b59c-8cb9bfdcc756 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bec0448f69ef4948b2190a2cff97bb04'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:37.097816 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-16893404-47d2-47dc-b59c-8cb9bfdcc756 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bec0448f69ef4948b2190a2cff97bb04'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:37.117245 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-69366071-b693-47b2-83ab-18fed160e631 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe9abbaf91794fa69709b1bf0c06d553', 'request_id': 'req-69366071-b693-47b2-83ab-18fed160e631'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:37.128277 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-16893404-47d2-47dc-b59c-8cb9bfdcc756 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bec0448f69ef4948b2190a2cff97bb04'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:37.135442 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-69366071-b693-47b2-83ab-18fed160e631 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe9abbaf91794fa69709b1bf0c06d553', 'request_id': 'req-69366071-b693-47b2-83ab-18fed160e631'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:37.142646 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 961/1916] 10.222.0.61 () {58 vars in 1224 bytes} [Mon Aug 22 19:19:36 2022] DELETE /identity/v3/users/bec0448f69ef4948b2190a2cff97bb04 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:19:37.146048 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-69366071-b693-47b2-83ab-18fed160e631 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe9abbaf91794fa69709b1bf0c06d553', 'request_id': 'req-69366071-b693-47b2-83ab-18fed160e631'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:37.151575 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94f1f05-ac2b-43a7-a0c7-c939328673d8 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:37.151896 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94f1f05-ac2b-43a7-a0c7-c939328673d8 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:37.152179 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94f1f05-ac2b-43a7-a0c7-c939328673d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:37.158759 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-69366071-b693-47b2-83ab-18fed160e631 admin admin] The token cache is being invalidate because project fe9abbaf91794fa69709b1bf0c06d553 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 19:19:37.159111 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-69366071-b693-47b2-83ab-18fed160e631 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:37.160155 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-69366071-b693-47b2-83ab-18fed160e631 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:37.161924 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 956/1917] 10.222.0.61 () {58 vars in 1233 bytes} [Mon Aug 22 19:19:37 2022] DELETE /identity/v3/projects/fe9abbaf91794fa69709b1bf0c06d553 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:19:37.165662 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69366071-b693-47b2-83ab-18fed160e631 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:37.167636 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-69366071-b693-47b2-83ab-18fed160e631 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:37.202872 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:19:37.204084 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-d94f1f05-ac2b-43a7-a0c7-c939328673d8 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:19:37.205354 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-d94f1f05-ac2b-43a7-a0c7-c939328673d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:37.251629 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69366071-b693-47b2-83ab-18fed160e631 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:37.321806 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870dc1d4-c668-4deb-a623-48e852113e26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:37.322453 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 962/1918] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:19:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:19:37.322568 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870dc1d4-c668-4deb-a623-48e852113e26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:37.322865 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870dc1d4-c668-4deb-a623-48e852113e26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:37.323146 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870dc1d4-c668-4deb-a623-48e852113e26 admin admin] PATH_INFO: `/v3/users/5c83bd4a0576403dba33b273cbeb66ed` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:37.324330 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870dc1d4-c668-4deb-a623-48e852113e26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c83bd4a0576403dba33b273cbeb66ed)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:37.326247 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870dc1d4-c668-4deb-a623-48e852113e26 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:37.332235 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94f1f05-ac2b-43a7-a0c7-c939328673d8 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:37.334540 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-d94f1f05-ac2b-43a7-a0c7-c939328673d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:37.388980 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-870dc1d4-c668-4deb-a623-48e852113e26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c83bd4a0576403dba33b273cbeb66ed'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:37.389963 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-870dc1d4-c668-4deb-a623-48e852113e26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c83bd4a0576403dba33b273cbeb66ed'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:37.398779 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-870dc1d4-c668-4deb-a623-48e852113e26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c83bd4a0576403dba33b273cbeb66ed'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:37.406208 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94f1f05-ac2b-43a7-a0c7-c939328673d8 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:37.409120 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-870dc1d4-c668-4deb-a623-48e852113e26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c83bd4a0576403dba33b273cbeb66ed'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:37.410928 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7e6cb5-90e7-4be3-8894-ba355ab7afb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:37.411763 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7e6cb5-90e7-4be3-8894-ba355ab7afb3 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:37.412045 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7e6cb5-90e7-4be3-8894-ba355ab7afb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:37.412317 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7e6cb5-90e7-4be3-8894-ba355ab7afb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:37.414174 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-aa7e6cb5-90e7-4be3-8894-ba355ab7afb3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:37.424118 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-870dc1d4-c668-4deb-a623-48e852113e26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c83bd4a0576403dba33b273cbeb66ed'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:37.430479 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 957/1919] 10.222.0.61 () {58 vars in 1224 bytes} [Mon Aug 22 19:19:37 2022] DELETE /identity/v3/users/5c83bd4a0576403dba33b273cbeb66ed => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:19:37.431253 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870dc1d4-c668-4deb-a623-48e852113e26 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:37.433583 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-870dc1d4-c668-4deb-a623-48e852113e26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:37.436388 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7e6cb5-90e7-4be3-8894-ba355ab7afb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:37.438453 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7e6cb5-90e7-4be3-8894-ba355ab7afb3 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:37.450659 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 963/1920] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:19:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 19:19:37.496043 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:19:37.517677 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870dc1d4-c668-4deb-a623-48e852113e26 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:37.524420 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e944fb3-c13d-4092-87a6-8cd6b6499ac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:37.525072 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e944fb3-c13d-4092-87a6-8cd6b6499ac0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:37.525349 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e944fb3-c13d-4092-87a6-8cd6b6499ac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:37.525622 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e944fb3-c13d-4092-87a6-8cd6b6499ac0 admin admin] PATH_INFO: `/v3/users/a477d44f9f8646a3a123421659323775` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:37.527319 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e944fb3-c13d-4092-87a6-8cd6b6499ac0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a477d44f9f8646a3a123421659323775)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:37.529307 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e944fb3-c13d-4092-87a6-8cd6b6499ac0 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:37.595771 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-8e944fb3-c13d-4092-87a6-8cd6b6499ac0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a477d44f9f8646a3a123421659323775'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:37.596519 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-8e944fb3-c13d-4092-87a6-8cd6b6499ac0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a477d44f9f8646a3a123421659323775'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:37.607277 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-8e944fb3-c13d-4092-87a6-8cd6b6499ac0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a477d44f9f8646a3a123421659323775'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:37.616875 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-8e944fb3-c13d-4092-87a6-8cd6b6499ac0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a477d44f9f8646a3a123421659323775'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:37.634802 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-8e944fb3-c13d-4092-87a6-8cd6b6499ac0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a477d44f9f8646a3a123421659323775'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:37.645692 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 958/1921] 10.222.0.61 () {58 vars in 1224 bytes} [Mon Aug 22 19:19:37 2022] DELETE /identity/v3/users/a477d44f9f8646a3a123421659323775 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:19:37.651556 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a180faf2-bd89-46a9-8b8e-32121f6a4410 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:37.652358 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a180faf2-bd89-46a9-8b8e-32121f6a4410 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:37.652707 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a180faf2-bd89-46a9-8b8e-32121f6a4410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:37.677370 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-a180faf2-bd89-46a9-8b8e-32121f6a4410 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:19:37.678619 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-a180faf2-bd89-46a9-8b8e-32121f6a4410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:37.739064 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:19:37.771416 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 964/1922] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:19:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:19:37.782603 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e944fb3-c13d-4092-87a6-8cd6b6499ac0 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:37.784339 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-8e944fb3-c13d-4092-87a6-8cd6b6499ac0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:37.789236 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a180faf2-bd89-46a9-8b8e-32121f6a4410 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:37.790960 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-a180faf2-bd89-46a9-8b8e-32121f6a4410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:37.807538 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a180faf2-bd89-46a9-8b8e-32121f6a4410 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:37.815976 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4238c34d-107a-4e8f-8cdc-0ad6c0b1fd48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:37.816815 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4238c34d-107a-4e8f-8cdc-0ad6c0b1fd48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:37.817187 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4238c34d-107a-4e8f-8cdc-0ad6c0b1fd48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:37.817589 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4238c34d-107a-4e8f-8cdc-0ad6c0b1fd48 admin admin] PATH_INFO: `/v3/projects/07088db66a404b62ba31975d711df4bf` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:37.822344 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4238c34d-107a-4e8f-8cdc-0ad6c0b1fd48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07088db66a404b62ba31975d711df4bf)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:37.825992 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4238c34d-107a-4e8f-8cdc-0ad6c0b1fd48 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:37.876348 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e944fb3-c13d-4092-87a6-8cd6b6499ac0 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:37.880885 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-4238c34d-107a-4e8f-8cdc-0ad6c0b1fd48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07088db66a404b62ba31975d711df4bf', 'request_id': 'req-4238c34d-107a-4e8f-8cdc-0ad6c0b1fd48'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:37.890584 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-4238c34d-107a-4e8f-8cdc-0ad6c0b1fd48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07088db66a404b62ba31975d711df4bf', 'request_id': 'req-4238c34d-107a-4e8f-8cdc-0ad6c0b1fd48'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:37.898614 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-4238c34d-107a-4e8f-8cdc-0ad6c0b1fd48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07088db66a404b62ba31975d711df4bf', 'request_id': 'req-4238c34d-107a-4e8f-8cdc-0ad6c0b1fd48'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:37.909876 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-4238c34d-107a-4e8f-8cdc-0ad6c0b1fd48 admin admin] The token cache is being invalidate because project 07088db66a404b62ba31975d711df4bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 19:19:37.910168 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-4238c34d-107a-4e8f-8cdc-0ad6c0b1fd48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:37.910824 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-4238c34d-107a-4e8f-8cdc-0ad6c0b1fd48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:37.912692 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 965/1923] 10.222.0.61 () {58 vars in 1233 bytes} [Mon Aug 22 19:19:37 2022] DELETE /identity/v3/projects/07088db66a404b62ba31975d711df4bf => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:19:37.917459 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4238c34d-107a-4e8f-8cdc-0ad6c0b1fd48 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:37.918777 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-4238c34d-107a-4e8f-8cdc-0ad6c0b1fd48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:37.933678 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6b8d63-c475-4111-b493-799b27d9818f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:37.934317 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6b8d63-c475-4111-b493-799b27d9818f service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:37.934592 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6b8d63-c475-4111-b493-799b27d9818f service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:37.934980 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6b8d63-c475-4111-b493-799b27d9818f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:37.936852 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-cc6b8d63-c475-4111-b493-799b27d9818f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:37.981699 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:19:37.988174 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4238c34d-107a-4e8f-8cdc-0ad6c0b1fd48 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:37.995945 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fe8817-39a2-41e5-ae2a-c5bd448bddfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:37.996590 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fe8817-39a2-41e5-ae2a-c5bd448bddfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:37.996867 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fe8817-39a2-41e5-ae2a-c5bd448bddfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:37.997139 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fe8817-39a2-41e5-ae2a-c5bd448bddfc admin admin] PATH_INFO: `/v3/projects/371250516b4143ceb2fd1bbb4eec6e1c` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:37.998612 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fe8817-39a2-41e5-ae2a-c5bd448bddfc admin admin] RBAC: Authorizing `identity:delete_project(project_id=371250516b4143ceb2fd1bbb4eec6e1c)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:38.000803 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fe8817-39a2-41e5-ae2a-c5bd448bddfc admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:38.010067 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6b8d63-c475-4111-b493-799b27d9818f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:38.012120 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6b8d63-c475-4111-b493-799b27d9818f service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:38.022380 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 959/1924] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:19:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:19:38.023473 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4043d6-bc1e-47e8-96fd-9cc0154458ff None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:38.023788 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4043d6-bc1e-47e8-96fd-9cc0154458ff None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:38.024063 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4043d6-bc1e-47e8-96fd-9cc0154458ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:38.046040 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-8c4043d6-bc1e-47e8-96fd-9cc0154458ff None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:19:38.047271 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-8c4043d6-bc1e-47e8-96fd-9cc0154458ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:38.056052 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-e6fe8817-39a2-41e5-ae2a-c5bd448bddfc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '371250516b4143ceb2fd1bbb4eec6e1c', 'request_id': 'req-e6fe8817-39a2-41e5-ae2a-c5bd448bddfc'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:38.065271 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-e6fe8817-39a2-41e5-ae2a-c5bd448bddfc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '371250516b4143ceb2fd1bbb4eec6e1c', 'request_id': 'req-e6fe8817-39a2-41e5-ae2a-c5bd448bddfc'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:38.073431 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-e6fe8817-39a2-41e5-ae2a-c5bd448bddfc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '371250516b4143ceb2fd1bbb4eec6e1c', 'request_id': 'req-e6fe8817-39a2-41e5-ae2a-c5bd448bddfc'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:38.084166 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:19:38.088553 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-e6fe8817-39a2-41e5-ae2a-c5bd448bddfc admin admin] The token cache is being invalidate because project 371250516b4143ceb2fd1bbb4eec6e1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 19:19:38.088553 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-e6fe8817-39a2-41e5-ae2a-c5bd448bddfc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:38.089188 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-e6fe8817-39a2-41e5-ae2a-c5bd448bddfc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:38.091291 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 966/1925] 10.222.0.61 () {58 vars in 1233 bytes} [Mon Aug 22 19:19:37 2022] DELETE /identity/v3/projects/371250516b4143ceb2fd1bbb4eec6e1c => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:19:38.095117 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fe8817-39a2-41e5-ae2a-c5bd448bddfc admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:38.097170 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-e6fe8817-39a2-41e5-ae2a-c5bd448bddfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:38.125763 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 960/1926] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:19:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:19:38.134594 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15163570-42f3-4ca2-bb78-9774cede48a3 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:38.134881 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15163570-42f3-4ca2-bb78-9774cede48a3 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:38.135169 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15163570-42f3-4ca2-bb78-9774cede48a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:38.155488 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-15163570-42f3-4ca2-bb78-9774cede48a3 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:19:38.157201 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-15163570-42f3-4ca2-bb78-9774cede48a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:38.168615 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 961/1927] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:19:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 19:19:38.170381 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15163570-42f3-4ca2-bb78-9774cede48a3 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:38.182557 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15163570-42f3-4ca2-bb78-9774cede48a3 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:38.188108 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fe8817-39a2-41e5-ae2a-c5bd448bddfc admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:38.189316 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba07520-1e79-491c-990b-4f26d11456ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:38.189931 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba07520-1e79-491c-990b-4f26d11456ac admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:38.190190 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba07520-1e79-491c-990b-4f26d11456ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:38.190454 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba07520-1e79-491c-990b-4f26d11456ac admin admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:38.191095 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba07520-1e79-491c-990b-4f26d11456ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:19:38.191390 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba07520-1e79-491c-990b-4f26d11456ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:38.193025 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba07520-1e79-491c-990b-4f26d11456ac admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:38.196735 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564709db-33ea-4831-9bbb-0df70243633d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:38.197632 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564709db-33ea-4831-9bbb-0df70243633d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:38.198664 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564709db-33ea-4831-9bbb-0df70243633d admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:38.199252 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564709db-33ea-4831-9bbb-0df70243633d admin admin] PATH_INFO: `/v3/users/8aaea49abc3e408495ba38aaee34e70b` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:38.200735 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564709db-33ea-4831-9bbb-0df70243633d admin admin] RBAC: Authorizing `identity:delete_user(user_id=8aaea49abc3e408495ba38aaee34e70b)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:38.205503 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564709db-33ea-4831-9bbb-0df70243633d admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:38.208221 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 962/1928] 10.222.0.61 () {58 vars in 1153 bytes} [Mon Aug 22 19:19:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 19:19:38.212493 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba07520-1e79-491c-990b-4f26d11456ac admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:38.237645 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba07520-1e79-491c-990b-4f26d11456ac admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:38.246096 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5674d540-19ff-4bd4-998a-69318876fcd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:38.246753 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5674d540-19ff-4bd4-998a-69318876fcd5 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:38.247064 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5674d540-19ff-4bd4-998a-69318876fcd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:38.247516 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5674d540-19ff-4bd4-998a-69318876fcd5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:38.248287 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5674d540-19ff-4bd4-998a-69318876fcd5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:38.251061 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5674d540-19ff-4bd4-998a-69318876fcd5 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:38.284577 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 963/1929] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 19:19:38 2022] POST /identity/v3/projects => generated 375 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:19:38.289996 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5674d540-19ff-4bd4-998a-69318876fcd5 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:38.306220 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-564709db-33ea-4831-9bbb-0df70243633d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8aaea49abc3e408495ba38aaee34e70b'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:38.310768 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-564709db-33ea-4831-9bbb-0df70243633d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8aaea49abc3e408495ba38aaee34e70b'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:38.325742 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-564709db-33ea-4831-9bbb-0df70243633d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8aaea49abc3e408495ba38aaee34e70b'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:38.339412 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-564709db-33ea-4831-9bbb-0df70243633d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8aaea49abc3e408495ba38aaee34e70b'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:38.378229 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-564709db-33ea-4831-9bbb-0df70243633d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8aaea49abc3e408495ba38aaee34e70b'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:38.389960 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5674d540-19ff-4bd4-998a-69318876fcd5 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:38.393498 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 967/1930] 10.222.0.61 () {58 vars in 1224 bytes} [Mon Aug 22 19:19:38 2022] DELETE /identity/v3/users/8aaea49abc3e408495ba38aaee34e70b => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 19:19:38.402898 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210be6b4-31a6-4ec5-b8bc-7b4d89d8dfb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:38.403843 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210be6b4-31a6-4ec5-b8bc-7b4d89d8dfb0 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:38.404124 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210be6b4-31a6-4ec5-b8bc-7b4d89d8dfb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:38.404506 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210be6b4-31a6-4ec5-b8bc-7b4d89d8dfb0 admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:38.405275 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210be6b4-31a6-4ec5-b8bc-7b4d89d8dfb0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:38.408207 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210be6b4-31a6-4ec5-b8bc-7b4d89d8dfb0 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:38.434483 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 964/1931] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 19:19:38 2022] POST /identity/v3/users => generated 288 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:19:38.439401 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564709db-33ea-4831-9bbb-0df70243633d admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:38.441368 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-564709db-33ea-4831-9bbb-0df70243633d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:38.514525 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564709db-33ea-4831-9bbb-0df70243633d admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:38.519205 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc55bf4-5306-423a-85e7-6c7b5197aa79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:38.519852 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc55bf4-5306-423a-85e7-6c7b5197aa79 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:38.520129 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc55bf4-5306-423a-85e7-6c7b5197aa79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:38.520402 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc55bf4-5306-423a-85e7-6c7b5197aa79 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:38.521050 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc55bf4-5306-423a-85e7-6c7b5197aa79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:38.522663 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc55bf4-5306-423a-85e7-6c7b5197aa79 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:38.529019 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210be6b4-31a6-4ec5-b8bc-7b4d89d8dfb0 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:38.531353 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-210be6b4-31a6-4ec5-b8bc-7b4d89d8dfb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:38.535222 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 968/1932] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:19:38 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 19:19:38.539193 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc55bf4-5306-423a-85e7-6c7b5197aa79 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:38.548164 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210be6b4-31a6-4ec5-b8bc-7b4d89d8dfb0 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:38.550414 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc55bf4-5306-423a-85e7-6c7b5197aa79 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:38.553722 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3cc968-5435-4bc6-8253-687dff7f4a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:38.554392 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3cc968-5435-4bc6-8253-687dff7f4a0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:38.554623 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3cc968-5435-4bc6-8253-687dff7f4a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:38.554999 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3cc968-5435-4bc6-8253-687dff7f4a0c admin admin] PATH_INFO: `/v3/projects/64b20c01ca6b4ce380f8276431cd604d` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:38.556350 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3cc968-5435-4bc6-8253-687dff7f4a0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=64b20c01ca6b4ce380f8276431cd604d)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:38.557606 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c548e3fe-81c5-4298-ab03-18d2a286a518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:38.558393 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c548e3fe-81c5-4298-ab03-18d2a286a518 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:38.558673 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c548e3fe-81c5-4298-ab03-18d2a286a518 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:38.558998 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3cc968-5435-4bc6-8253-687dff7f4a0c admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:38.559106 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c548e3fe-81c5-4298-ab03-18d2a286a518 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:38.560795 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c548e3fe-81c5-4298-ab03-18d2a286a518 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:38.563442 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c548e3fe-81c5-4298-ab03-18d2a286a518 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:38.578674 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 969/1933] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:19:38 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 19:19:38.583017 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c548e3fe-81c5-4298-ab03-18d2a286a518 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:38.590331 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:19:38.595926 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c548e3fe-81c5-4298-ab03-18d2a286a518 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:38.602702 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6b5fb7-8727-43a4-8d83-3e6193531f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:38.603745 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6b5fb7-8727-43a4-8d83-3e6193531f54 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:38.604640 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6b5fb7-8727-43a4-8d83-3e6193531f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:38.604988 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6b5fb7-8727-43a4-8d83-3e6193531f54 admin admin] PATH_INFO: `/v3/projects/1e9f4c09721e4206869bbc0e49f9ab50/users/01a14c36af46451884400a5b8db1702d/roles/74a40bc819cb42eea7e966a4b2a016f1` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:38.606308 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:19:38.628159 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6b5fb7-8727-43a4-8d83-3e6193531f54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e9f4c09721e4206869bbc0e49f9ab50, user_id=01a14c36af46451884400a5b8db1702d, role_id=74a40bc819cb42eea7e966a4b2a016f1)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:38.631738 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6b5fb7-8727-43a4-8d83-3e6193531f54 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:38.646056 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-cf3cc968-5435-4bc6-8253-687dff7f4a0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64b20c01ca6b4ce380f8276431cd604d', 'request_id': 'req-cf3cc968-5435-4bc6-8253-687dff7f4a0c'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:38.651832 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 970/1934] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:19:38 2022] PUT /identity/v3/projects/1e9f4c09721e4206869bbc0e49f9ab50/users/01a14c36af46451884400a5b8db1702d/roles/74a40bc819cb42eea7e966a4b2a016f1 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 19:19:38.654593 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-cf3cc968-5435-4bc6-8253-687dff7f4a0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64b20c01ca6b4ce380f8276431cd604d', 'request_id': 'req-cf3cc968-5435-4bc6-8253-687dff7f4a0c'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:38.661134 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ec9ca4-8d3f-4fb0-96da-617ae63ea92a None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:38.661537 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ec9ca4-8d3f-4fb0-96da-617ae63ea92a None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:38.661756 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ec9ca4-8d3f-4fb0-96da-617ae63ea92a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:38.662601 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-cf3cc968-5435-4bc6-8253-687dff7f4a0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64b20c01ca6b4ce380f8276431cd604d', 'request_id': 'req-cf3cc968-5435-4bc6-8253-687dff7f4a0c'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:38.675440 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-cf3cc968-5435-4bc6-8253-687dff7f4a0c admin admin] The token cache is being invalidate because project 64b20c01ca6b4ce380f8276431cd604d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 19:19:38.675813 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-cf3cc968-5435-4bc6-8253-687dff7f4a0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:38.676489 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-cf3cc968-5435-4bc6-8253-687dff7f4a0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:38.678845 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 965/1935] 10.222.0.61 () {58 vars in 1233 bytes} [Mon Aug 22 19:19:38 2022] DELETE /identity/v3/projects/64b20c01ca6b4ce380f8276431cd604d => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:19:38.683214 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3cc968-5435-4bc6-8253-687dff7f4a0c admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:38.684337 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-cf3cc968-5435-4bc6-8253-687dff7f4a0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:38.694136 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-a3ec9ca4-8d3f-4fb0-96da-617ae63ea92a None None] MFA Rules not processed for user `01a14c36af46451884400a5b8db1702d`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:19:38.695336 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-a3ec9ca4-8d3f-4fb0-96da-617ae63ea92a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:38.765046 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3cc968-5435-4bc6-8253-687dff7f4a0c admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:38.787874 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 971/1936] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:19:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:19:38.800714 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ec9ca4-8d3f-4fb0-96da-617ae63ea92a None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:38.802316 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-a3ec9ca4-8d3f-4fb0-96da-617ae63ea92a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:38.832085 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf92db9-030f-4b86-a3f9-f08405fc2a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:38.832863 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf92db9-030f-4b86-a3f9-f08405fc2a45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:38.833146 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf92db9-030f-4b86-a3f9-f08405fc2a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:38.833589 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf92db9-030f-4b86-a3f9-f08405fc2a45 admin admin] PATH_INFO: `/v3/users/570f8f1038ca474bba31f467e74a17d0` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:38.835048 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf92db9-030f-4b86-a3f9-f08405fc2a45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=570f8f1038ca474bba31f467e74a17d0)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:38.837078 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf92db9-030f-4b86-a3f9-f08405fc2a45 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:38.874040 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ec9ca4-8d3f-4fb0-96da-617ae63ea92a None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:38.878647 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aa603d-cf7d-42f9-bfe8-0b4d14ba3eb9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e79498873d4ed29a0e19bbea9b9214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:38.879271 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aa603d-cf7d-42f9-bfe8-0b4d14ba3eb9 service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:38.879834 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aa603d-cf7d-42f9-bfe8-0b4d14ba3eb9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:38.880123 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aa603d-cf7d-42f9-bfe8-0b4d14ba3eb9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:38.889100 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34aa603d-cf7d-42f9-bfe8-0b4d14ba3eb9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:38.890583 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34aa603d-cf7d-42f9-bfe8-0b4d14ba3eb9 service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:38.897443 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-7bf92db9-030f-4b86-a3f9-f08405fc2a45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '570f8f1038ca474bba31f467e74a17d0'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:38.898112 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-7bf92db9-030f-4b86-a3f9-f08405fc2a45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '570f8f1038ca474bba31f467e74a17d0'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:38.899554 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 972/1937] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:19:38 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 19:19:38.900515 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aa603d-cf7d-42f9-bfe8-0b4d14ba3eb9 service cinder] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:38.902909 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-34aa603d-cf7d-42f9-bfe8-0b4d14ba3eb9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:38.911633 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-7bf92db9-030f-4b86-a3f9-f08405fc2a45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '570f8f1038ca474bba31f467e74a17d0'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:38.913697 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:19:38.919036 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aa603d-cf7d-42f9-bfe8-0b4d14ba3eb9 service cinder] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:38.923955 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-7bf92db9-030f-4b86-a3f9-f08405fc2a45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '570f8f1038ca474bba31f467e74a17d0'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:38.927085 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ad2dcf-f2cb-4911-acda-db7693572b88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:38.927939 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ad2dcf-f2cb-4911-acda-db7693572b88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:38.928398 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ad2dcf-f2cb-4911-acda-db7693572b88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:38.928671 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ad2dcf-f2cb-4911-acda-db7693572b88 admin admin] PATH_INFO: `/v3/projects/0a506d8d74844f9fb369ed96977e68e4` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:38.932294 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ad2dcf-f2cb-4911-acda-db7693572b88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a506d8d74844f9fb369ed96977e68e4)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:38.934527 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ad2dcf-f2cb-4911-acda-db7693572b88 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:38.939799 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-7bf92db9-030f-4b86-a3f9-f08405fc2a45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '570f8f1038ca474bba31f467e74a17d0'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:38.947279 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 966/1938] 10.222.0.61 () {58 vars in 1224 bytes} [Mon Aug 22 19:19:38 2022] DELETE /identity/v3/users/570f8f1038ca474bba31f467e74a17d0 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 19:19:38.993665 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-89ad2dcf-f2cb-4911-acda-db7693572b88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a506d8d74844f9fb369ed96977e68e4', 'request_id': 'req-89ad2dcf-f2cb-4911-acda-db7693572b88'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:39.005411 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-89ad2dcf-f2cb-4911-acda-db7693572b88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a506d8d74844f9fb369ed96977e68e4', 'request_id': 'req-89ad2dcf-f2cb-4911-acda-db7693572b88'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:39.014533 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-89ad2dcf-f2cb-4911-acda-db7693572b88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a506d8d74844f9fb369ed96977e68e4', 'request_id': 'req-89ad2dcf-f2cb-4911-acda-db7693572b88'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:39.021357 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:19:39.029424 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-89ad2dcf-f2cb-4911-acda-db7693572b88 admin admin] The token cache is being invalidate because project 0a506d8d74844f9fb369ed96977e68e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 19:19:39.029720 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-89ad2dcf-f2cb-4911-acda-db7693572b88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:39.031936 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-89ad2dcf-f2cb-4911-acda-db7693572b88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:39.033908 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 973/1939] 10.222.0.61 () {58 vars in 1233 bytes} [Mon Aug 22 19:19:38 2022] DELETE /identity/v3/projects/0a506d8d74844f9fb369ed96977e68e4 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:19:39.038836 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18915930-cf17-47f3-b651-dfdabf9ee3ec None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:39.039137 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18915930-cf17-47f3-b651-dfdabf9ee3ec None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:39.039405 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18915930-cf17-47f3-b651-dfdabf9ee3ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:39.062594 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-18915930-cf17-47f3-b651-dfdabf9ee3ec None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:19:39.063852 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-18915930-cf17-47f3-b651-dfdabf9ee3ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:39.147478 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a305165d-e7dc-408a-a22f-9e363ac373b7 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:39.147478 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a305165d-e7dc-408a-a22f-9e363ac373b7 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:39.147478 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a305165d-e7dc-408a-a22f-9e363ac373b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:39.152270 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 967/1940] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:19:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 19:19:39.186611 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-a305165d-e7dc-408a-a22f-9e363ac373b7 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:19:39.188592 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-a305165d-e7dc-408a-a22f-9e363ac373b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:39.199271 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 974/1941] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:19:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:19:39.203459 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18915930-cf17-47f3-b651-dfdabf9ee3ec None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:39.222553 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18915930-cf17-47f3-b651-dfdabf9ee3ec None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:39.231329 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18333310-74ba-4dff-ab1d-90762e3c2652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:39.235785 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18333310-74ba-4dff-ab1d-90762e3c2652 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:39.236120 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18333310-74ba-4dff-ab1d-90762e3c2652 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:39.236547 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18333310-74ba-4dff-ab1d-90762e3c2652 admin admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:39.237624 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18333310-74ba-4dff-ab1d-90762e3c2652 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:19:39.237935 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18333310-74ba-4dff-ab1d-90762e3c2652 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:39.241310 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18333310-74ba-4dff-ab1d-90762e3c2652 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:39.256329 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 968/1942] 10.222.0.61 () {58 vars in 1153 bytes} [Mon Aug 22 19:19:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:19:39.261052 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a305165d-e7dc-408a-a22f-9e363ac373b7 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:39.272877 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a305165d-e7dc-408a-a22f-9e363ac373b7 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:39.282355 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017a5af9-eccd-4ab5-a5f1-3cbddcafeaf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:39.283196 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017a5af9-eccd-4ab5-a5f1-3cbddcafeaf3 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:39.283704 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017a5af9-eccd-4ab5-a5f1-3cbddcafeaf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:39.284173 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017a5af9-eccd-4ab5-a5f1-3cbddcafeaf3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:39.284973 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017a5af9-eccd-4ab5-a5f1-3cbddcafeaf3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:39.287174 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017a5af9-eccd-4ab5-a5f1-3cbddcafeaf3 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:39.322227 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 975/1943] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 19:19:39 2022] POST /identity/v3/projects => generated 383 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:19:39.327258 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18333310-74ba-4dff-ab1d-90762e3c2652 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:39.338005 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18333310-74ba-4dff-ab1d-90762e3c2652 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:39.397067 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402838c5-5e69-499b-bc1a-9515a7a6d4cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:39.397681 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402838c5-5e69-499b-bc1a-9515a7a6d4cc admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:39.397953 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402838c5-5e69-499b-bc1a-9515a7a6d4cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:39.398222 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402838c5-5e69-499b-bc1a-9515a7a6d4cc admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:39.398845 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402838c5-5e69-499b-bc1a-9515a7a6d4cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:39.401345 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402838c5-5e69-499b-bc1a-9515a7a6d4cc admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:39.427268 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 969/1944] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 19:19:39 2022] POST /identity/v3/users => generated 292 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:19:39.430346 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017a5af9-eccd-4ab5-a5f1-3cbddcafeaf3 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:39.445276 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017a5af9-eccd-4ab5-a5f1-3cbddcafeaf3 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:39.455396 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e97668e-f6cb-4bc1-9b84-23e855e42766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:39.456161 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e97668e-f6cb-4bc1-9b84-23e855e42766 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:39.456533 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e97668e-f6cb-4bc1-9b84-23e855e42766 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:39.456991 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e97668e-f6cb-4bc1-9b84-23e855e42766 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:39.457886 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e97668e-f6cb-4bc1-9b84-23e855e42766 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:39.465639 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e97668e-f6cb-4bc1-9b84-23e855e42766 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:39.477819 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 976/1945] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:19:39 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:19:39.483455 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402838c5-5e69-499b-bc1a-9515a7a6d4cc admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:39.485282 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e97668e-f6cb-4bc1-9b84-23e855e42766 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:39.492452 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-7e97668e-f6cb-4bc1-9b84-23e855e42766 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:39.506713 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e97668e-f6cb-4bc1-9b84-23e855e42766 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:39.512569 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76170aaa-abfc-4b1b-a8fa-031dc8e3918c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:39.513337 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76170aaa-abfc-4b1b-a8fa-031dc8e3918c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:39.513619 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76170aaa-abfc-4b1b-a8fa-031dc8e3918c admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:39.513892 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76170aaa-abfc-4b1b-a8fa-031dc8e3918c admin admin] PATH_INFO: `/v3/projects/3b4f0102ce5749fe82b66a3943b0df76` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:39.515207 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76170aaa-abfc-4b1b-a8fa-031dc8e3918c admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b4f0102ce5749fe82b66a3943b0df76)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:39.517077 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76170aaa-abfc-4b1b-a8fa-031dc8e3918c admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:39.523117 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402838c5-5e69-499b-bc1a-9515a7a6d4cc admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:39.531936 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8de81d-6958-4b1b-964c-fae6afebe1d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:39.532565 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8de81d-6958-4b1b-964c-fae6afebe1d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:39.532849 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8de81d-6958-4b1b-964c-fae6afebe1d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:39.533121 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8de81d-6958-4b1b-964c-fae6afebe1d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:39.533807 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8de81d-6958-4b1b-964c-fae6afebe1d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:39.535409 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8de81d-6958-4b1b-964c-fae6afebe1d0 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:39.552787 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 970/1946] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:19:39 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:19:39.556858 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8de81d-6958-4b1b-964c-fae6afebe1d0 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:39.570289 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8de81d-6958-4b1b-964c-fae6afebe1d0 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:39.578626 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-76170aaa-abfc-4b1b-a8fa-031dc8e3918c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b4f0102ce5749fe82b66a3943b0df76', 'request_id': 'req-76170aaa-abfc-4b1b-a8fa-031dc8e3918c'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:39.587081 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-76170aaa-abfc-4b1b-a8fa-031dc8e3918c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b4f0102ce5749fe82b66a3943b0df76', 'request_id': 'req-76170aaa-abfc-4b1b-a8fa-031dc8e3918c'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:39.594922 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-76170aaa-abfc-4b1b-a8fa-031dc8e3918c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b4f0102ce5749fe82b66a3943b0df76', 'request_id': 'req-76170aaa-abfc-4b1b-a8fa-031dc8e3918c'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:39.606385 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-76170aaa-abfc-4b1b-a8fa-031dc8e3918c admin admin] The token cache is being invalidate because project 3b4f0102ce5749fe82b66a3943b0df76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 19:19:39.606744 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-76170aaa-abfc-4b1b-a8fa-031dc8e3918c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:39.607501 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-76170aaa-abfc-4b1b-a8fa-031dc8e3918c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:39.609301 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 977/1947] 10.222.0.61 () {58 vars in 1233 bytes} [Mon Aug 22 19:19:39 2022] DELETE /identity/v3/projects/3b4f0102ce5749fe82b66a3943b0df76 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:19:39.613680 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2027b5-95b2-4234-9018-3550f029bdf1 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:39.613950 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2027b5-95b2-4234-9018-3550f029bdf1 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:39.614221 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2027b5-95b2-4234-9018-3550f029bdf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:39.618231 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:19:39.639840 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-2c2027b5-95b2-4234-9018-3550f029bdf1 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:19:39.641122 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-2c2027b5-95b2-4234-9018-3550f029bdf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:39.641427 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e73800c-c851-4eb8-a37d-42680e96ebcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:39.642235 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e73800c-c851-4eb8-a37d-42680e96ebcc admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:39.642624 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e73800c-c851-4eb8-a37d-42680e96ebcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:39.642917 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e73800c-c851-4eb8-a37d-42680e96ebcc admin admin] PATH_INFO: `/v3/projects/cd7e34cfc21b4b9c9ccc992faaadb75f/users/13452ef446614d13a81f83f101cb69b8/roles/74a40bc819cb42eea7e966a4b2a016f1` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:39.649941 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 978/1948] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:19:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 19:19:39.660212 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e73800c-c851-4eb8-a37d-42680e96ebcc admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd7e34cfc21b4b9c9ccc992faaadb75f, user_id=13452ef446614d13a81f83f101cb69b8, role_id=74a40bc819cb42eea7e966a4b2a016f1)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:39.660348 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca0cd76-91e7-44d8-bb16-d068f44f37a9 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:39.660630 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca0cd76-91e7-44d8-bb16-d068f44f37a9 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:39.661046 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca0cd76-91e7-44d8-bb16-d068f44f37a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:39.664293 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e73800c-c851-4eb8-a37d-42680e96ebcc admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:39.677920 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 971/1949] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:19:39 2022] PUT /identity/v3/projects/cd7e34cfc21b4b9c9ccc992faaadb75f/users/13452ef446614d13a81f83f101cb69b8/roles/74a40bc819cb42eea7e966a4b2a016f1 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 19:19:39.685945 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-9ca0cd76-91e7-44d8-bb16-d068f44f37a9 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:19:39.687382 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-9ca0cd76-91e7-44d8-bb16-d068f44f37a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:39.689407 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158ebca5-0960-4f0c-bc26-85553822ee6e None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:39.689695 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158ebca5-0960-4f0c-bc26-85553822ee6e None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:39.689974 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158ebca5-0960-4f0c-bc26-85553822ee6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:39.719855 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-158ebca5-0960-4f0c-bc26-85553822ee6e None None] MFA Rules not processed for user `13452ef446614d13a81f83f101cb69b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:19:39.721086 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-158ebca5-0960-4f0c-bc26-85553822ee6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:39.762024 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 979/1950] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:19:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 19:19:39.766279 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca0cd76-91e7-44d8-bb16-d068f44f37a9 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:39.776598 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca0cd76-91e7-44d8-bb16-d068f44f37a9 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:39.781242 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8271c7aa-2d66-4cc6-9bae-8848a2daf074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:39.781857 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8271c7aa-2d66-4cc6-9bae-8848a2daf074 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:39.782131 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8271c7aa-2d66-4cc6-9bae-8848a2daf074 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:39.782402 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8271c7aa-2d66-4cc6-9bae-8848a2daf074 admin admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:39.783341 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8271c7aa-2d66-4cc6-9bae-8848a2daf074 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:19:39.783672 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8271c7aa-2d66-4cc6-9bae-8848a2daf074 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:39.785799 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8271c7aa-2d66-4cc6-9bae-8848a2daf074 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:39.793592 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 972/1951] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:19:39 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 19:19:39.799748 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 980/1952] 10.222.0.61 () {58 vars in 1153 bytes} [Mon Aug 22 19:19:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 19:19:39.803549 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158ebca5-0960-4f0c-bc26-85553822ee6e None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:39.805915 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8271c7aa-2d66-4cc6-9bae-8848a2daf074 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:39.808899 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-8271c7aa-2d66-4cc6-9bae-8848a2daf074 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:39.825123 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158ebca5-0960-4f0c-bc26-85553822ee6e None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:39.829766 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b83561-926f-461c-950d-caf2559c7372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:39.830368 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b83561-926f-461c-950d-caf2559c7372 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:39.830639 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b83561-926f-461c-950d-caf2559c7372 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:39.830906 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b83561-926f-461c-950d-caf2559c7372 admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:39.831528 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b83561-926f-461c-950d-caf2559c7372 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:39.832655 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:19:39.833611 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b83561-926f-461c-950d-caf2559c7372 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:39.862849 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 973/1953] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 19:19:39 2022] POST /identity/v3/projects => generated 373 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 19:19:39.867007 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b83561-926f-461c-950d-caf2559c7372 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:39.872653 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8271c7aa-2d66-4cc6-9bae-8848a2daf074 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:39.919277 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15213dc5-9494-42f4-b4e0-c6c2bfcb954f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e79498873d4ed29a0e19bbea9b9214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:39.920117 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15213dc5-9494-42f4-b4e0-c6c2bfcb954f service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:39.920392 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15213dc5-9494-42f4-b4e0-c6c2bfcb954f service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:39.920654 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15213dc5-9494-42f4-b4e0-c6c2bfcb954f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:39.929649 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b83561-926f-461c-950d-caf2559c7372 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:39.935062 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b751d5c-64be-4640-9e76-c3f4cdba5946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:39.935689 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b751d5c-64be-4640-9e76-c3f4cdba5946 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:39.935977 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b751d5c-64be-4640-9e76-c3f4cdba5946 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:39.936254 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b751d5c-64be-4640-9e76-c3f4cdba5946 admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:39.936878 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b751d5c-64be-4640-9e76-c3f4cdba5946 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:39.938785 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b751d5c-64be-4640-9e76-c3f4cdba5946 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:39.962633 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 974/1954] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 19:19:39 2022] POST /identity/v3/users => generated 287 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 19:19:39.966908 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b751d5c-64be-4640-9e76-c3f4cdba5946 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:39.976323 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b751d5c-64be-4640-9e76-c3f4cdba5946 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:39.979599 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15213dc5-9494-42f4-b4e0-c6c2bfcb954f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:39.981094 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15213dc5-9494-42f4-b4e0-c6c2bfcb954f service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:39.981775 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbbbb8f-b4e6-43f5-92cc-f2b6b511a4a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:39.982401 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbbbb8f-b4e6-43f5-92cc-f2b6b511a4a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:39.982680 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbbbb8f-b4e6-43f5-92cc-f2b6b511a4a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:39.982953 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbbbb8f-b4e6-43f5-92cc-f2b6b511a4a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:39.983591 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbbbb8f-b4e6-43f5-92cc-f2b6b511a4a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:39.985196 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbbbb8f-b4e6-43f5-92cc-f2b6b511a4a2 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:39.990510 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 981/1955] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:19:39 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:19:39.998512 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 975/1956] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:19:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 19:19:39.999826 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15213dc5-9494-42f4-b4e0-c6c2bfcb954f service cinder] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:40.010296 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15213dc5-9494-42f4-b4e0-c6c2bfcb954f service cinder] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:40.014831 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23639168-6343-4f97-9f8d-3477097d3a37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:40.015427 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23639168-6343-4f97-9f8d-3477097d3a37 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:40.015733 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23639168-6343-4f97-9f8d-3477097d3a37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:40.016004 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23639168-6343-4f97-9f8d-3477097d3a37 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:40.016631 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23639168-6343-4f97-9f8d-3477097d3a37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:40.018206 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23639168-6343-4f97-9f8d-3477097d3a37 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:40.028146 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 982/1957] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:19:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:19:40.032121 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbbbb8f-b4e6-43f5-92cc-f2b6b511a4a2 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:40.042750 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbbbb8f-b4e6-43f5-92cc-f2b6b511a4a2 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:40.047864 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf295d9b-e14d-44de-b21a-20fe46eb10ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:40.048645 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf295d9b-e14d-44de-b21a-20fe46eb10ae admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:40.048936 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf295d9b-e14d-44de-b21a-20fe46eb10ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:40.049216 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf295d9b-e14d-44de-b21a-20fe46eb10ae admin admin] PATH_INFO: `/v3/projects/787a5d42e81f4f689739df3970857e69/users/552ef606db694455bacc36f27b7c6bae/roles/74a40bc819cb42eea7e966a4b2a016f1` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:40.052569 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:19:40.066921 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf295d9b-e14d-44de-b21a-20fe46eb10ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=787a5d42e81f4f689739df3970857e69, user_id=552ef606db694455bacc36f27b7c6bae, role_id=74a40bc819cb42eea7e966a4b2a016f1)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:40.069603 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf295d9b-e14d-44de-b21a-20fe46eb10ae admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:40.084287 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 976/1958] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:19:40 2022] PUT /identity/v3/projects/787a5d42e81f4f689739df3970857e69/users/552ef606db694455bacc36f27b7c6bae/roles/74a40bc819cb42eea7e966a4b2a016f1 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:19:40.095849 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7510746-9486-4e52-9d5f-841be9b4cc01 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:40.096126 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7510746-9486-4e52-9d5f-841be9b4cc01 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:40.096402 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7510746-9486-4e52-9d5f-841be9b4cc01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:40.130554 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-e7510746-9486-4e52-9d5f-841be9b4cc01 None None] MFA Rules not processed for user `552ef606db694455bacc36f27b7c6bae`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:19:40.132541 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-e7510746-9486-4e52-9d5f-841be9b4cc01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:40.221815 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 983/1959] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:19:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:19:40.226090 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf295d9b-e14d-44de-b21a-20fe46eb10ae admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:40.328167 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf295d9b-e14d-44de-b21a-20fe46eb10ae admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:40.341506 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0c0d0f-dcff-4197-8335-b20f3b3db20a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:40.343889 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0c0d0f-dcff-4197-8335-b20f3b3db20a admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:40.344361 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0c0d0f-dcff-4197-8335-b20f3b3db20a admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:40.345273 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0c0d0f-dcff-4197-8335-b20f3b3db20a admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:40.347056 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0c0d0f-dcff-4197-8335-b20f3b3db20a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:40.351139 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0c0d0f-dcff-4197-8335-b20f3b3db20a admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:40.407624 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 977/1960] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 19:19:40 2022] POST /identity/v3/projects => generated 373 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 19:19:40.408777 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7510746-9486-4e52-9d5f-841be9b4cc01 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:40.420968 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7510746-9486-4e52-9d5f-841be9b4cc01 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:40.426608 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:19:40.490031 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b834df9-a97a-41f2-9ab5-faec1c659116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:40.490679 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b834df9-a97a-41f2-9ab5-faec1c659116 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:40.490959 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b834df9-a97a-41f2-9ab5-faec1c659116 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:40.491233 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b834df9-a97a-41f2-9ab5-faec1c659116 admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:40.492051 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b834df9-a97a-41f2-9ab5-faec1c659116 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:40.494310 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b834df9-a97a-41f2-9ab5-faec1c659116 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:40.523629 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 984/1961] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 19:19:40 2022] POST /identity/v3/users => generated 293 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:19:40.524840 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0c0d0f-dcff-4197-8335-b20f3b3db20a admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:40.536009 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0c0d0f-dcff-4197-8335-b20f3b3db20a admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:40.540521 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa845ce-ba3b-4257-af80-2558e0856beb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:40.541123 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa845ce-ba3b-4257-af80-2558e0856beb admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:40.542070 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa845ce-ba3b-4257-af80-2558e0856beb admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:40.542344 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa845ce-ba3b-4257-af80-2558e0856beb admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:40.542989 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa845ce-ba3b-4257-af80-2558e0856beb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:40.544603 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa845ce-ba3b-4257-af80-2558e0856beb admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:40.555566 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 978/1962] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:19:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:19:40.559898 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b834df9-a97a-41f2-9ab5-faec1c659116 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:40.570268 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b834df9-a97a-41f2-9ab5-faec1c659116 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:40.576091 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0b6307-0cf4-44fc-b1dc-0779dcbb6d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:40.576755 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0b6307-0cf4-44fc-b1dc-0779dcbb6d8b admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:40.577031 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0b6307-0cf4-44fc-b1dc-0779dcbb6d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:40.577308 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0b6307-0cf4-44fc-b1dc-0779dcbb6d8b admin admin] PATH_INFO: `/v3/projects/9138020696204c2395762ddc97749a0a/users/8c78868799564eb1990441c0e7ad67a6/roles/163c30c3b3c246beb849ce176f02d9da` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:40.593020 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0b6307-0cf4-44fc-b1dc-0779dcbb6d8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9138020696204c2395762ddc97749a0a, user_id=8c78868799564eb1990441c0e7ad67a6, role_id=163c30c3b3c246beb849ce176f02d9da)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:40.595674 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0b6307-0cf4-44fc-b1dc-0779dcbb6d8b admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:40.611285 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 985/1963] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:19:40 2022] PUT /identity/v3/projects/9138020696204c2395762ddc97749a0a/users/8c78868799564eb1990441c0e7ad67a6/roles/163c30c3b3c246beb849ce176f02d9da => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 19:19:40.615368 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa845ce-ba3b-4257-af80-2558e0856beb admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:40.627364 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa845ce-ba3b-4257-af80-2558e0856beb admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:40.696780 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f205d94-6316-446c-901b-78f3ebf0bd98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:40.697395 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f205d94-6316-446c-901b-78f3ebf0bd98 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:40.698221 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f205d94-6316-446c-901b-78f3ebf0bd98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:40.698493 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f205d94-6316-446c-901b-78f3ebf0bd98 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:40.699137 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f205d94-6316-446c-901b-78f3ebf0bd98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:40.700752 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f205d94-6316-446c-901b-78f3ebf0bd98 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:40.710754 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 979/1964] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:19:40 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:19:40.715478 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0b6307-0cf4-44fc-b1dc-0779dcbb6d8b admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:40.725763 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0b6307-0cf4-44fc-b1dc-0779dcbb6d8b admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:40.730707 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45450b8f-3e24-4cb9-9b1f-43de7fa98935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:40.731354 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45450b8f-3e24-4cb9-9b1f-43de7fa98935 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:40.731635 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45450b8f-3e24-4cb9-9b1f-43de7fa98935 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:40.731932 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45450b8f-3e24-4cb9-9b1f-43de7fa98935 admin admin] PATH_INFO: `/v3/projects/9138020696204c2395762ddc97749a0a/users/8c78868799564eb1990441c0e7ad67a6/roles/163c30c3b3c246beb849ce176f02d9da` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:40.733834 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45450b8f-3e24-4cb9-9b1f-43de7fa98935 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9138020696204c2395762ddc97749a0a, user_id=8c78868799564eb1990441c0e7ad67a6, role_id=163c30c3b3c246beb849ce176f02d9da)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:40.736495 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45450b8f-3e24-4cb9-9b1f-43de7fa98935 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:40.748980 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 986/1965] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:19:40 2022] PUT /identity/v3/projects/9138020696204c2395762ddc97749a0a/users/8c78868799564eb1990441c0e7ad67a6/roles/163c30c3b3c246beb849ce176f02d9da => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 19:19:40.759102 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293064d4-25e3-4f4d-ba2f-727f25faaf0e None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:40.759386 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293064d4-25e3-4f4d-ba2f-727f25faaf0e None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:40.759684 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293064d4-25e3-4f4d-ba2f-727f25faaf0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:40.793669 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-293064d4-25e3-4f4d-ba2f-727f25faaf0e None None] MFA Rules not processed for user `8c78868799564eb1990441c0e7ad67a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:19:40.794886 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-293064d4-25e3-4f4d-ba2f-727f25faaf0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:40.837787 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:19:40.863685 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 980/1966] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:19:40 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:19:40.875063 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45450b8f-3e24-4cb9-9b1f-43de7fa98935 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:40.930843 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45450b8f-3e24-4cb9-9b1f-43de7fa98935 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:40.934688 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc74ddf-caec-4729-aee9-ef69409c67e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e79498873d4ed29a0e19bbea9b9214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:40.935330 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc74ddf-caec-4729-aee9-ef69409c67e1 service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:40.935607 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc74ddf-caec-4729-aee9-ef69409c67e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:40.935913 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc74ddf-caec-4729-aee9-ef69409c67e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:40.945116 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc74ddf-caec-4729-aee9-ef69409c67e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:40.946544 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc74ddf-caec-4729-aee9-ef69409c67e1 service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:40.956642 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 987/1967] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:19:40 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:19:41.337746 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293064d4-25e3-4f4d-ba2f-727f25faaf0e None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:41.406410 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293064d4-25e3-4f4d-ba2f-727f25faaf0e None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:41.411763 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f483e2e7-f637-4ccb-bc0b-c69e6ac50a44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:41.412416 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f483e2e7-f637-4ccb-bc0b-c69e6ac50a44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:41.412698 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f483e2e7-f637-4ccb-bc0b-c69e6ac50a44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:41.412972 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f483e2e7-f637-4ccb-bc0b-c69e6ac50a44 admin admin] PATH_INFO: `/v3/users/552ef606db694455bacc36f27b7c6bae` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:41.414404 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f483e2e7-f637-4ccb-bc0b-c69e6ac50a44 admin admin] RBAC: Authorizing `identity:delete_user(user_id=552ef606db694455bacc36f27b7c6bae)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:41.416154 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f483e2e7-f637-4ccb-bc0b-c69e6ac50a44 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:41.473162 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-f483e2e7-f637-4ccb-bc0b-c69e6ac50a44 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '552ef606db694455bacc36f27b7c6bae'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:41.474487 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-f483e2e7-f637-4ccb-bc0b-c69e6ac50a44 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '552ef606db694455bacc36f27b7c6bae'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:41.482865 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-f483e2e7-f637-4ccb-bc0b-c69e6ac50a44 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '552ef606db694455bacc36f27b7c6bae'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:41.491181 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-f483e2e7-f637-4ccb-bc0b-c69e6ac50a44 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '552ef606db694455bacc36f27b7c6bae'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:41.504244 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-f483e2e7-f637-4ccb-bc0b-c69e6ac50a44 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '552ef606db694455bacc36f27b7c6bae'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:41.510480 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 981/1968] 10.222.0.61 () {58 vars in 1224 bytes} [Mon Aug 22 19:19:41 2022] DELETE /identity/v3/users/552ef606db694455bacc36f27b7c6bae => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 19:19:41.514948 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e60ed40-4a7c-4251-9c7e-227d5653a4bf None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:41.515236 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e60ed40-4a7c-4251-9c7e-227d5653a4bf None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:41.515506 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e60ed40-4a7c-4251-9c7e-227d5653a4bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:41.534054 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-5e60ed40-4a7c-4251-9c7e-227d5653a4bf None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:19:41.535264 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-5e60ed40-4a7c-4251-9c7e-227d5653a4bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:41.599001 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 988/1969] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:19:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 19:19:41.609551 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f483e2e7-f637-4ccb-bc0b-c69e6ac50a44 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:41.610953 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-f483e2e7-f637-4ccb-bc0b-c69e6ac50a44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:41.668586 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f483e2e7-f637-4ccb-bc0b-c69e6ac50a44 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:41.672171 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a97320-1846-4e3c-bd7b-135224801041 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:41.672776 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a97320-1846-4e3c-bd7b-135224801041 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:41.673053 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a97320-1846-4e3c-bd7b-135224801041 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:41.673324 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a97320-1846-4e3c-bd7b-135224801041 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:41.683024 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a97320-1846-4e3c-bd7b-135224801041 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:41.684515 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a97320-1846-4e3c-bd7b-135224801041 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:41.694448 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 982/1970] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:19:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:19:42.100956 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e60ed40-4a7c-4251-9c7e-227d5653a4bf None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:42.102778 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-5e60ed40-4a7c-4251-9c7e-227d5653a4bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:42.111908 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:19:42.116309 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e60ed40-4a7c-4251-9c7e-227d5653a4bf None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:42.123752 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138db11a-4755-4abf-bd50-bbc277d19702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:42.124463 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138db11a-4755-4abf-bd50-bbc277d19702 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:42.124671 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138db11a-4755-4abf-bd50-bbc277d19702 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:42.124946 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138db11a-4755-4abf-bd50-bbc277d19702 admin admin] PATH_INFO: `/v3/projects/787a5d42e81f4f689739df3970857e69` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:42.126093 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138db11a-4755-4abf-bd50-bbc277d19702 admin admin] RBAC: Authorizing `identity:delete_project(project_id=787a5d42e81f4f689739df3970857e69)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:42.132544 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138db11a-4755-4abf-bd50-bbc277d19702 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:42.187818 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-138db11a-4755-4abf-bd50-bbc277d19702 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '787a5d42e81f4f689739df3970857e69', 'request_id': 'req-138db11a-4755-4abf-bd50-bbc277d19702'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:42.201245 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-138db11a-4755-4abf-bd50-bbc277d19702 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '787a5d42e81f4f689739df3970857e69', 'request_id': 'req-138db11a-4755-4abf-bd50-bbc277d19702'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:42.207670 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-138db11a-4755-4abf-bd50-bbc277d19702 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '787a5d42e81f4f689739df3970857e69', 'request_id': 'req-138db11a-4755-4abf-bd50-bbc277d19702'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:42.218599 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-138db11a-4755-4abf-bd50-bbc277d19702 admin admin] The token cache is being invalidate because project 787a5d42e81f4f689739df3970857e69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 19:19:42.218880 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-138db11a-4755-4abf-bd50-bbc277d19702 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:42.219526 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-138db11a-4755-4abf-bd50-bbc277d19702 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:42.221101 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 989/1971] 10.222.0.61 () {58 vars in 1233 bytes} [Mon Aug 22 19:19:42 2022] DELETE /identity/v3/projects/787a5d42e81f4f689739df3970857e69 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:19:42.225112 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a97320-1846-4e3c-bd7b-135224801041 service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:42.230252 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:19:42.236465 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a97320-1846-4e3c-bd7b-135224801041 service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:42.293982 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e562dc7-adf2-449f-95dd-dfece92b0592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:42.294610 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e562dc7-adf2-449f-95dd-dfece92b0592 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:42.294888 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e562dc7-adf2-449f-95dd-dfece92b0592 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:42.295159 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e562dc7-adf2-449f-95dd-dfece92b0592 admin admin] PATH_INFO: `/v3/users/8c78868799564eb1990441c0e7ad67a6` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:42.296300 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e562dc7-adf2-449f-95dd-dfece92b0592 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c78868799564eb1990441c0e7ad67a6)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:42.298200 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e562dc7-adf2-449f-95dd-dfece92b0592 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:42.342224 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-7e562dc7-adf2-449f-95dd-dfece92b0592 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c78868799564eb1990441c0e7ad67a6'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:42.342869 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-7e562dc7-adf2-449f-95dd-dfece92b0592 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c78868799564eb1990441c0e7ad67a6'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:42.349663 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-7e562dc7-adf2-449f-95dd-dfece92b0592 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c78868799564eb1990441c0e7ad67a6'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:42.356976 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-7e562dc7-adf2-449f-95dd-dfece92b0592 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c78868799564eb1990441c0e7ad67a6'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:42.369095 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-7e562dc7-adf2-449f-95dd-dfece92b0592 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c78868799564eb1990441c0e7ad67a6'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:42.375444 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 983/1972] 10.222.0.61 () {58 vars in 1224 bytes} [Mon Aug 22 19:19:42 2022] DELETE /identity/v3/users/8c78868799564eb1990441c0e7ad67a6 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:19:42.652431 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138db11a-4755-4abf-bd50-bbc277d19702 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:42.653905 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-138db11a-4755-4abf-bd50-bbc277d19702 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:42.669272 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138db11a-4755-4abf-bd50-bbc277d19702 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:42.731046 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d21932-fe92-49cb-bec0-6152195f8eb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:42.731806 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d21932-fe92-49cb-bec0-6152195f8eb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:42.732086 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d21932-fe92-49cb-bec0-6152195f8eb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:42.732359 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d21932-fe92-49cb-bec0-6152195f8eb0 admin admin] PATH_INFO: `/v3/projects/9138020696204c2395762ddc97749a0a` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:42.733503 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d21932-fe92-49cb-bec0-6152195f8eb0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9138020696204c2395762ddc97749a0a)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:42.735422 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d21932-fe92-49cb-bec0-6152195f8eb0 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:42.783890 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-14d21932-fe92-49cb-bec0-6152195f8eb0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9138020696204c2395762ddc97749a0a', 'request_id': 'req-14d21932-fe92-49cb-bec0-6152195f8eb0'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:42.793292 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-14d21932-fe92-49cb-bec0-6152195f8eb0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9138020696204c2395762ddc97749a0a', 'request_id': 'req-14d21932-fe92-49cb-bec0-6152195f8eb0'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:42.799723 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-14d21932-fe92-49cb-bec0-6152195f8eb0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9138020696204c2395762ddc97749a0a', 'request_id': 'req-14d21932-fe92-49cb-bec0-6152195f8eb0'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:42.810919 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-14d21932-fe92-49cb-bec0-6152195f8eb0 admin admin] The token cache is being invalidate because project 9138020696204c2395762ddc97749a0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 19:19:42.811208 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-14d21932-fe92-49cb-bec0-6152195f8eb0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:42.811884 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-14d21932-fe92-49cb-bec0-6152195f8eb0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:42.813383 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 990/1973] 10.222.0.61 () {58 vars in 1233 bytes} [Mon Aug 22 19:19:42 2022] DELETE /identity/v3/projects/9138020696204c2395762ddc97749a0a => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:19:42.819222 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a351e9b-aa9b-46b8-9fc9-75994da5a10d None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:42.819731 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a351e9b-aa9b-46b8-9fc9-75994da5a10d None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:42.819843 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a351e9b-aa9b-46b8-9fc9-75994da5a10d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:42.840309 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-2a351e9b-aa9b-46b8-9fc9-75994da5a10d None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:19:42.841650 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-2a351e9b-aa9b-46b8-9fc9-75994da5a10d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:42.911294 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 984/1974] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:19:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:19:42.919957 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df593b3f-93f6-4b32-a45e-10d84bdc6dca None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:42.920194 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df593b3f-93f6-4b32-a45e-10d84bdc6dca None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:42.920437 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df593b3f-93f6-4b32-a45e-10d84bdc6dca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:42.941239 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-df593b3f-93f6-4b32-a45e-10d84bdc6dca None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:19:42.942476 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-df593b3f-93f6-4b32-a45e-10d84bdc6dca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:42.950875 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 991/1975] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:19:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:19:42.954895 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a351e9b-aa9b-46b8-9fc9-75994da5a10d None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:42.964811 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a351e9b-aa9b-46b8-9fc9-75994da5a10d None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:42.969468 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba020368-ad96-4ada-9a9e-110bd800d529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:42.970076 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba020368-ad96-4ada-9a9e-110bd800d529 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:42.970355 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba020368-ad96-4ada-9a9e-110bd800d529 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:42.970624 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba020368-ad96-4ada-9a9e-110bd800d529 admin admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:42.971282 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba020368-ad96-4ada-9a9e-110bd800d529 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:19:42.971581 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba020368-ad96-4ada-9a9e-110bd800d529 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:42.973196 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba020368-ad96-4ada-9a9e-110bd800d529 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:42.986311 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 985/1976] 10.222.0.61 () {58 vars in 1153 bytes} [Mon Aug 22 19:19:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:19:42.990752 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df593b3f-93f6-4b32-a45e-10d84bdc6dca None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:43.001151 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df593b3f-93f6-4b32-a45e-10d84bdc6dca None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:43.006060 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaf687f-cd13-455d-9c81-95cd33766c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:43.006679 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaf687f-cd13-455d-9c81-95cd33766c66 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:43.006962 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaf687f-cd13-455d-9c81-95cd33766c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:43.007237 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaf687f-cd13-455d-9c81-95cd33766c66 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:43.008221 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaf687f-cd13-455d-9c81-95cd33766c66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:43.010754 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaf687f-cd13-455d-9c81-95cd33766c66 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:43.022474 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:19:43.055907 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 992/1977] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 19:19:42 2022] POST /identity/v3/projects => generated 367 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 19:19:43.058793 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba020368-ad96-4ada-9a9e-110bd800d529 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:43.071111 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba020368-ad96-4ada-9a9e-110bd800d529 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:43.077548 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:19:43.152594 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8a7c0c-a7fd-4715-a854-9527d8e65c6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:43.153435 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8a7c0c-a7fd-4715-a854-9527d8e65c6d admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:43.153774 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8a7c0c-a7fd-4715-a854-9527d8e65c6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:43.154014 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8a7c0c-a7fd-4715-a854-9527d8e65c6d admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:43.154778 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8a7c0c-a7fd-4715-a854-9527d8e65c6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:43.158084 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8a7c0c-a7fd-4715-a854-9527d8e65c6d admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:43.198416 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 986/1978] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 19:19:43 2022] POST /identity/v3/users => generated 284 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:19:43.200081 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaf687f-cd13-455d-9c81-95cd33766c66 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:43.212542 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaf687f-cd13-455d-9c81-95cd33766c66 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:43.219310 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbecad27-a2e5-4a5a-a16c-6ab10750d2ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:43.220050 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbecad27-a2e5-4a5a-a16c-6ab10750d2ce admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:43.220287 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbecad27-a2e5-4a5a-a16c-6ab10750d2ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:43.220619 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbecad27-a2e5-4a5a-a16c-6ab10750d2ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:43.221509 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbecad27-a2e5-4a5a-a16c-6ab10750d2ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:43.223107 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbecad27-a2e5-4a5a-a16c-6ab10750d2ce admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:43.236364 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 993/1979] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:19:43 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 19:19:43.240284 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8a7c0c-a7fd-4715-a854-9527d8e65c6d admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:43.253281 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8a7c0c-a7fd-4715-a854-9527d8e65c6d admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:43.260920 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63327d6-ee69-4a2d-8d7e-c8d1b48de82a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:43.261512 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63327d6-ee69-4a2d-8d7e-c8d1b48de82a admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:43.261819 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63327d6-ee69-4a2d-8d7e-c8d1b48de82a admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:43.262320 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63327d6-ee69-4a2d-8d7e-c8d1b48de82a admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:43.262934 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63327d6-ee69-4a2d-8d7e-c8d1b48de82a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:43.264561 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63327d6-ee69-4a2d-8d7e-c8d1b48de82a admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:43.276660 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 987/1980] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:19:43 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:19:43.280903 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbecad27-a2e5-4a5a-a16c-6ab10750d2ce admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:43.294677 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbecad27-a2e5-4a5a-a16c-6ab10750d2ce admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:43.299755 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508d2fa6-b4fa-4716-9dac-80b5a170265c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:43.300423 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508d2fa6-b4fa-4716-9dac-80b5a170265c admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:43.300701 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508d2fa6-b4fa-4716-9dac-80b5a170265c admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:43.300973 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508d2fa6-b4fa-4716-9dac-80b5a170265c admin admin] PATH_INFO: `/v3/projects/e43278fe261741d992c3087fbc04fea3/users/04324c0022d84a5bb852e7d71c91e56d/roles/74a40bc819cb42eea7e966a4b2a016f1` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:43.318161 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508d2fa6-b4fa-4716-9dac-80b5a170265c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e43278fe261741d992c3087fbc04fea3, user_id=04324c0022d84a5bb852e7d71c91e56d, role_id=74a40bc819cb42eea7e966a4b2a016f1)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:43.321024 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508d2fa6-b4fa-4716-9dac-80b5a170265c admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:43.339239 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 994/1981] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:19:43 2022] PUT /identity/v3/projects/e43278fe261741d992c3087fbc04fea3/users/04324c0022d84a5bb852e7d71c91e56d/roles/74a40bc819cb42eea7e966a4b2a016f1 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:19:43.350365 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcc1577-b849-43cd-8290-6d55d76792f8 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:43.350648 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcc1577-b849-43cd-8290-6d55d76792f8 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:43.350918 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcc1577-b849-43cd-8290-6d55d76792f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:43.380401 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-edcc1577-b849-43cd-8290-6d55d76792f8 None None] MFA Rules not processed for user `04324c0022d84a5bb852e7d71c91e56d`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:19:43.381664 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-edcc1577-b849-43cd-8290-6d55d76792f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:43.444812 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 988/1982] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:19:43 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:19:43.449305 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508d2fa6-b4fa-4716-9dac-80b5a170265c admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:43.513184 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508d2fa6-b4fa-4716-9dac-80b5a170265c admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:43.517798 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b4c7e4-c4f5-4687-94e1-4da1dda45dfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:43.518409 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b4c7e4-c4f5-4687-94e1-4da1dda45dfc admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:43.518687 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b4c7e4-c4f5-4687-94e1-4da1dda45dfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:43.518951 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b4c7e4-c4f5-4687-94e1-4da1dda45dfc admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:43.519575 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b4c7e4-c4f5-4687-94e1-4da1dda45dfc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:43.521480 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b4c7e4-c4f5-4687-94e1-4da1dda45dfc admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:43.545330 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de17cf1a-b903-421a-9c95-c6eebb524ff9 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:43.545936 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de17cf1a-b903-421a-9c95-c6eebb524ff9 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:43.546562 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de17cf1a-b903-421a-9c95-c6eebb524ff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:43.550256 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 995/1983] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 19:19:43 2022] POST /identity/v3/projects => generated 367 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:19:43.554144 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b4c7e4-c4f5-4687-94e1-4da1dda45dfc admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:43.572423 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-de17cf1a-b903-421a-9c95-c6eebb524ff9 None None] MFA Rules not processed for user `01a14c36af46451884400a5b8db1702d`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:19:43.575124 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-de17cf1a-b903-421a-9c95-c6eebb524ff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:43.630206 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b4c7e4-c4f5-4687-94e1-4da1dda45dfc admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:43.634870 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267603c0-59c0-4507-8674-780028653915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:43.635503 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267603c0-59c0-4507-8674-780028653915 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:43.635820 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267603c0-59c0-4507-8674-780028653915 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:43.636094 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267603c0-59c0-4507-8674-780028653915 admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:43.636712 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267603c0-59c0-4507-8674-780028653915 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:43.638631 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267603c0-59c0-4507-8674-780028653915 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:43.638740 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 989/1984] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:19:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:19:43.660342 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 996/1985] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 19:19:43 2022] POST /identity/v3/users => generated 290 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 19:19:43.664441 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de17cf1a-b903-421a-9c95-c6eebb524ff9 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:43.676319 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de17cf1a-b903-421a-9c95-c6eebb524ff9 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:43.680995 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b566535e-aef5-4532-b70a-3b5fb704e1e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:43.681584 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b566535e-aef5-4532-b70a-3b5fb704e1e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:43.681865 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b566535e-aef5-4532-b70a-3b5fb704e1e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:43.682699 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b566535e-aef5-4532-b70a-3b5fb704e1e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:43.683354 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b566535e-aef5-4532-b70a-3b5fb704e1e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:43.685008 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b566535e-aef5-4532-b70a-3b5fb704e1e3 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:43.699785 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 990/1986] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:19:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 19:19:43.700714 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267603c0-59c0-4507-8674-780028653915 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:43.711073 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267603c0-59c0-4507-8674-780028653915 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:43.715630 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5b6d91-af08-4044-bf90-8830d2120a93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:43.716323 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5b6d91-af08-4044-bf90-8830d2120a93 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:43.716600 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5b6d91-af08-4044-bf90-8830d2120a93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:43.716866 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5b6d91-af08-4044-bf90-8830d2120a93 admin admin] PATH_INFO: `/v3/projects/d87085476ee44fdca8bd784d2f9fb37d/users/c948e019dadc48dc872851a3429bcf67/roles/163c30c3b3c246beb849ce176f02d9da` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:43.734662 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5b6d91-af08-4044-bf90-8830d2120a93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d87085476ee44fdca8bd784d2f9fb37d, user_id=c948e019dadc48dc872851a3429bcf67, role_id=163c30c3b3c246beb849ce176f02d9da)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:43.737408 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5b6d91-af08-4044-bf90-8830d2120a93 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:43.772356 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 997/1987] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:19:43 2022] PUT /identity/v3/projects/d87085476ee44fdca8bd784d2f9fb37d/users/c948e019dadc48dc872851a3429bcf67/roles/163c30c3b3c246beb849ce176f02d9da => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 19:19:43.776705 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b566535e-aef5-4532-b70a-3b5fb704e1e3 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:43.791112 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b566535e-aef5-4532-b70a-3b5fb704e1e3 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:43.821671 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:19:43.917026 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea99fd89-6fa4-409f-9dc6-1474d5fc6ae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:43.917675 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea99fd89-6fa4-409f-9dc6-1474d5fc6ae2 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:43.917948 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea99fd89-6fa4-409f-9dc6-1474d5fc6ae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:43.918262 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea99fd89-6fa4-409f-9dc6-1474d5fc6ae2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:43.919426 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea99fd89-6fa4-409f-9dc6-1474d5fc6ae2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:43.922239 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea99fd89-6fa4-409f-9dc6-1474d5fc6ae2 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:43.946070 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 991/1988] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:19:43 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:19:43.947562 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5b6d91-af08-4044-bf90-8830d2120a93 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:43.955993 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:19:43.966346 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5b6d91-af08-4044-bf90-8830d2120a93 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:43.982889 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5c4488-0d15-4d7e-89be-3a41fe959a56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:43.983592 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5c4488-0d15-4d7e-89be-3a41fe959a56 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:43.983910 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5c4488-0d15-4d7e-89be-3a41fe959a56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:43.984181 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5c4488-0d15-4d7e-89be-3a41fe959a56 admin admin] PATH_INFO: `/v3/projects/d87085476ee44fdca8bd784d2f9fb37d/users/c948e019dadc48dc872851a3429bcf67/roles/163c30c3b3c246beb849ce176f02d9da` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:43.988121 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5c4488-0d15-4d7e-89be-3a41fe959a56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d87085476ee44fdca8bd784d2f9fb37d, user_id=c948e019dadc48dc872851a3429bcf67, role_id=163c30c3b3c246beb849ce176f02d9da)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:43.990880 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5c4488-0d15-4d7e-89be-3a41fe959a56 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:44.005088 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 998/1989] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:19:43 2022] PUT /identity/v3/projects/d87085476ee44fdca8bd784d2f9fb37d/users/c948e019dadc48dc872851a3429bcf67/roles/163c30c3b3c246beb849ce176f02d9da => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:19:44.016606 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e656155-9b6d-47be-a508-fb9e1b6b407f None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:44.017872 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e656155-9b6d-47be-a508-fb9e1b6b407f None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:44.018202 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e656155-9b6d-47be-a508-fb9e1b6b407f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:44.056309 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-7e656155-9b6d-47be-a508-fb9e1b6b407f None None] MFA Rules not processed for user `c948e019dadc48dc872851a3429bcf67`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:19:44.058630 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-7e656155-9b6d-47be-a508-fb9e1b6b407f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:44.156315 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 992/1990] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:19:44 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:19:44.168518 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5c4488-0d15-4d7e-89be-3a41fe959a56 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:44.170858 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-3c5c4488-0d15-4d7e-89be-3a41fe959a56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:44.240935 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5c4488-0d15-4d7e-89be-3a41fe959a56 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:44.245336 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b203f7a4-49ab-4c10-9352-14734b26f0a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e79498873d4ed29a0e19bbea9b9214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:44.246369 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b203f7a4-49ab-4c10-9352-14734b26f0a9 service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:44.246369 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b203f7a4-49ab-4c10-9352-14734b26f0a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:44.246599 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b203f7a4-49ab-4c10-9352-14734b26f0a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:44.267736 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b203f7a4-49ab-4c10-9352-14734b26f0a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:44.267736 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b203f7a4-49ab-4c10-9352-14734b26f0a9 service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:44.268933 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 999/1991] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:19:44 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:19:45.911202 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e656155-9b6d-47be-a508-fb9e1b6b407f None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:45.971988 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e656155-9b6d-47be-a508-fb9e1b6b407f None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:45.976436 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3974b8fc-45a2-4c81-88c0-54aacea134f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:45.977311 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3974b8fc-45a2-4c81-88c0-54aacea134f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:45.977586 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3974b8fc-45a2-4c81-88c0-54aacea134f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:45.977853 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3974b8fc-45a2-4c81-88c0-54aacea134f9 admin admin] PATH_INFO: `/v3/users/04324c0022d84a5bb852e7d71c91e56d` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:45.978967 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3974b8fc-45a2-4c81-88c0-54aacea134f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=04324c0022d84a5bb852e7d71c91e56d)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:45.980883 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3974b8fc-45a2-4c81-88c0-54aacea134f9 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:46.017307 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:19:46.027899 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-3974b8fc-45a2-4c81-88c0-54aacea134f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04324c0022d84a5bb852e7d71c91e56d'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:46.028542 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-3974b8fc-45a2-4c81-88c0-54aacea134f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04324c0022d84a5bb852e7d71c91e56d'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:46.035605 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-3974b8fc-45a2-4c81-88c0-54aacea134f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04324c0022d84a5bb852e7d71c91e56d'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:46.042147 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-3974b8fc-45a2-4c81-88c0-54aacea134f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04324c0022d84a5bb852e7d71c91e56d'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:46.054116 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-3974b8fc-45a2-4c81-88c0-54aacea134f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04324c0022d84a5bb852e7d71c91e56d'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:46.060019 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 993/1992] 10.222.0.61 () {58 vars in 1224 bytes} [Mon Aug 22 19:19:45 2022] DELETE /identity/v3/users/04324c0022d84a5bb852e7d71c91e56d => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:19:46.064492 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76f61d9-c294-46cb-bc47-161d2e55f1bf None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:46.064949 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76f61d9-c294-46cb-bc47-161d2e55f1bf None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:46.065059 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76f61d9-c294-46cb-bc47-161d2e55f1bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:46.084074 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-f76f61d9-c294-46cb-bc47-161d2e55f1bf None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:19:46.085262 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-f76f61d9-c294-46cb-bc47-161d2e55f1bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:46.144059 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1000/1993] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:19:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:19:46.154435 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3974b8fc-45a2-4c81-88c0-54aacea134f9 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:46.155889 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-3974b8fc-45a2-4c81-88c0-54aacea134f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:46.212065 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3974b8fc-45a2-4c81-88c0-54aacea134f9 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:46.216291 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fb45db-c4d0-461a-a549-711296221b1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:46.216898 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fb45db-c4d0-461a-a549-711296221b1d service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:46.217175 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fb45db-c4d0-461a-a549-711296221b1d service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:46.217451 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fb45db-c4d0-461a-a549-711296221b1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:46.226335 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fb45db-c4d0-461a-a549-711296221b1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:46.228060 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fb45db-c4d0-461a-a549-711296221b1d service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:46.237977 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 994/1994] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:19:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:19:46.509222 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76f61d9-c294-46cb-bc47-161d2e55f1bf None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:46.510667 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-f76f61d9-c294-46cb-bc47-161d2e55f1bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:46.522971 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76f61d9-c294-46cb-bc47-161d2e55f1bf None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:46.528313 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596faee1-3b19-4ba0-bb90-80f1c7bf2000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:46.529047 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596faee1-3b19-4ba0-bb90-80f1c7bf2000 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:46.529497 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596faee1-3b19-4ba0-bb90-80f1c7bf2000 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:46.529838 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596faee1-3b19-4ba0-bb90-80f1c7bf2000 admin admin] PATH_INFO: `/v3/projects/e43278fe261741d992c3087fbc04fea3` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:46.531328 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596faee1-3b19-4ba0-bb90-80f1c7bf2000 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e43278fe261741d992c3087fbc04fea3)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:46.533283 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596faee1-3b19-4ba0-bb90-80f1c7bf2000 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:46.580713 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-596faee1-3b19-4ba0-bb90-80f1c7bf2000 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e43278fe261741d992c3087fbc04fea3', 'request_id': 'req-596faee1-3b19-4ba0-bb90-80f1c7bf2000'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:46.590504 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-596faee1-3b19-4ba0-bb90-80f1c7bf2000 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e43278fe261741d992c3087fbc04fea3', 'request_id': 'req-596faee1-3b19-4ba0-bb90-80f1c7bf2000'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:46.597042 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-596faee1-3b19-4ba0-bb90-80f1c7bf2000 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e43278fe261741d992c3087fbc04fea3', 'request_id': 'req-596faee1-3b19-4ba0-bb90-80f1c7bf2000'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:46.608713 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-596faee1-3b19-4ba0-bb90-80f1c7bf2000 admin admin] The token cache is being invalidate because project e43278fe261741d992c3087fbc04fea3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 19:19:46.609001 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-596faee1-3b19-4ba0-bb90-80f1c7bf2000 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:46.610114 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-596faee1-3b19-4ba0-bb90-80f1c7bf2000 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:46.611503 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1001/1995] 10.222.0.61 () {58 vars in 1233 bytes} [Mon Aug 22 19:19:46 2022] DELETE /identity/v3/projects/e43278fe261741d992c3087fbc04fea3 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:19:46.615263 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fb45db-c4d0-461a-a549-711296221b1d service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:46.628710 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fb45db-c4d0-461a-a549-711296221b1d service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:46.686842 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e4cbbd-a40c-4345-a79b-ad758b874124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:46.687481 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e4cbbd-a40c-4345-a79b-ad758b874124 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:46.687879 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e4cbbd-a40c-4345-a79b-ad758b874124 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:46.688064 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e4cbbd-a40c-4345-a79b-ad758b874124 admin admin] PATH_INFO: `/v3/users/c948e019dadc48dc872851a3429bcf67` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:46.689198 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e4cbbd-a40c-4345-a79b-ad758b874124 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c948e019dadc48dc872851a3429bcf67)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:46.691107 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e4cbbd-a40c-4345-a79b-ad758b874124 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:46.740227 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-83e4cbbd-a40c-4345-a79b-ad758b874124 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c948e019dadc48dc872851a3429bcf67'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:46.740885 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-83e4cbbd-a40c-4345-a79b-ad758b874124 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c948e019dadc48dc872851a3429bcf67'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:46.749053 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-83e4cbbd-a40c-4345-a79b-ad758b874124 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c948e019dadc48dc872851a3429bcf67'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:46.751206 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:19:46.759522 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-83e4cbbd-a40c-4345-a79b-ad758b874124 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c948e019dadc48dc872851a3429bcf67'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:46.772784 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-83e4cbbd-a40c-4345-a79b-ad758b874124 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c948e019dadc48dc872851a3429bcf67'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:46.779699 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 995/1996] 10.222.0.61 () {58 vars in 1224 bytes} [Mon Aug 22 19:19:46 2022] DELETE /identity/v3/users/c948e019dadc48dc872851a3429bcf67 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 19:19:47.144339 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596faee1-3b19-4ba0-bb90-80f1c7bf2000 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:47.145881 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-596faee1-3b19-4ba0-bb90-80f1c7bf2000 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:47.160714 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596faee1-3b19-4ba0-bb90-80f1c7bf2000 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:47.174690 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:19:47.220823 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e741cb5-4fa9-49f2-a39e-e16a7440f174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:47.221462 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e741cb5-4fa9-49f2-a39e-e16a7440f174 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:47.221739 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e741cb5-4fa9-49f2-a39e-e16a7440f174 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:47.222013 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e741cb5-4fa9-49f2-a39e-e16a7440f174 admin admin] PATH_INFO: `/v3/projects/d87085476ee44fdca8bd784d2f9fb37d` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:47.223133 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e741cb5-4fa9-49f2-a39e-e16a7440f174 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d87085476ee44fdca8bd784d2f9fb37d)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:47.225067 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e741cb5-4fa9-49f2-a39e-e16a7440f174 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:47.270071 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-1e741cb5-4fa9-49f2-a39e-e16a7440f174 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd87085476ee44fdca8bd784d2f9fb37d', 'request_id': 'req-1e741cb5-4fa9-49f2-a39e-e16a7440f174'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:47.277045 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-1e741cb5-4fa9-49f2-a39e-e16a7440f174 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd87085476ee44fdca8bd784d2f9fb37d', 'request_id': 'req-1e741cb5-4fa9-49f2-a39e-e16a7440f174'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:47.283065 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-1e741cb5-4fa9-49f2-a39e-e16a7440f174 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd87085476ee44fdca8bd784d2f9fb37d', 'request_id': 'req-1e741cb5-4fa9-49f2-a39e-e16a7440f174'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:47.293909 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-1e741cb5-4fa9-49f2-a39e-e16a7440f174 admin admin] The token cache is being invalidate because project d87085476ee44fdca8bd784d2f9fb37d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 19:19:47.294199 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-1e741cb5-4fa9-49f2-a39e-e16a7440f174 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:47.294823 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-1e741cb5-4fa9-49f2-a39e-e16a7440f174 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:47.296351 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1002/1997] 10.222.0.61 () {58 vars in 1233 bytes} [Mon Aug 22 19:19:47 2022] DELETE /identity/v3/projects/d87085476ee44fdca8bd784d2f9fb37d => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:19:47.301205 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3ea4f1-2324-40fa-a008-23cc7ffaf99b None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:47.301691 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3ea4f1-2324-40fa-a008-23cc7ffaf99b None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:47.301797 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3ea4f1-2324-40fa-a008-23cc7ffaf99b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:47.320722 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-ac3ea4f1-2324-40fa-a008-23cc7ffaf99b None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:19:47.321928 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-ac3ea4f1-2324-40fa-a008-23cc7ffaf99b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:47.380285 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 996/1998] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:19:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:19:47.388694 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7d50ba-0f54-4d2d-8d05-58dbe0965284 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:47.388979 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7d50ba-0f54-4d2d-8d05-58dbe0965284 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:47.389276 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7d50ba-0f54-4d2d-8d05-58dbe0965284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:47.409053 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-ca7d50ba-0f54-4d2d-8d05-58dbe0965284 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:19:47.410260 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-ca7d50ba-0f54-4d2d-8d05-58dbe0965284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:47.418079 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1003/1999] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:19:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:19:47.422211 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3ea4f1-2324-40fa-a008-23cc7ffaf99b None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:47.436832 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3ea4f1-2324-40fa-a008-23cc7ffaf99b None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:47.441425 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13e440d-9cea-47f9-977d-404d9ac5cd63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:47.442030 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13e440d-9cea-47f9-977d-404d9ac5cd63 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:47.442308 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13e440d-9cea-47f9-977d-404d9ac5cd63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:47.442586 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13e440d-9cea-47f9-977d-404d9ac5cd63 admin admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:47.443241 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13e440d-9cea-47f9-977d-404d9ac5cd63 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:19:47.443539 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13e440d-9cea-47f9-977d-404d9ac5cd63 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:47.445160 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13e440d-9cea-47f9-977d-404d9ac5cd63 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:47.457000 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 997/2000] 10.222.0.61 () {58 vars in 1153 bytes} [Mon Aug 22 19:19:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:19:47.461412 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7d50ba-0f54-4d2d-8d05-58dbe0965284 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:47.471604 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7d50ba-0f54-4d2d-8d05-58dbe0965284 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:47.476282 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce46089-0e5f-467b-8238-0f09073adf4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:47.477175 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce46089-0e5f-467b-8238-0f09073adf4a admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:47.477450 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce46089-0e5f-467b-8238-0f09073adf4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:47.477716 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce46089-0e5f-467b-8238-0f09073adf4a admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:47.478329 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce46089-0e5f-467b-8238-0f09073adf4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:47.480282 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce46089-0e5f-467b-8238-0f09073adf4a admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:47.511471 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1004/2001] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 19:19:47 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:19:47.512781 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13e440d-9cea-47f9-977d-404d9ac5cd63 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:47.523031 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13e440d-9cea-47f9-977d-404d9ac5cd63 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:47.582035 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1556ca7b-56f2-4891-9076-9afd758b62c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:47.582651 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1556ca7b-56f2-4891-9076-9afd758b62c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:47.582931 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1556ca7b-56f2-4891-9076-9afd758b62c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:47.583205 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1556ca7b-56f2-4891-9076-9afd758b62c5 admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:47.584159 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1556ca7b-56f2-4891-9076-9afd758b62c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:47.586117 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1556ca7b-56f2-4891-9076-9afd758b62c5 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:47.607847 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 998/2002] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 19:19:47 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:19:47.611918 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce46089-0e5f-467b-8238-0f09073adf4a admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:47.622095 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce46089-0e5f-467b-8238-0f09073adf4a admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:47.626594 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3631b77b-c248-4529-9a1d-c5ee04caf3f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:47.627186 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3631b77b-c248-4529-9a1d-c5ee04caf3f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:47.627461 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3631b77b-c248-4529-9a1d-c5ee04caf3f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:47.627768 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3631b77b-c248-4529-9a1d-c5ee04caf3f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:47.628407 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3631b77b-c248-4529-9a1d-c5ee04caf3f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:47.629979 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3631b77b-c248-4529-9a1d-c5ee04caf3f6 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:47.640663 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1005/2003] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:19:47 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:19:47.646472 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1556ca7b-56f2-4891-9076-9afd758b62c5 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:47.656872 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1556ca7b-56f2-4891-9076-9afd758b62c5 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:47.661447 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ab3d1e-a89b-45a6-a2e6-71220a4bd52f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:47.662061 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ab3d1e-a89b-45a6-a2e6-71220a4bd52f admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:47.662319 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ab3d1e-a89b-45a6-a2e6-71220a4bd52f admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:47.662587 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ab3d1e-a89b-45a6-a2e6-71220a4bd52f admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:47.663225 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ab3d1e-a89b-45a6-a2e6-71220a4bd52f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:47.664835 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ab3d1e-a89b-45a6-a2e6-71220a4bd52f admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:47.675223 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 999/2004] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:19:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:19:47.679525 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3631b77b-c248-4529-9a1d-c5ee04caf3f6 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:47.690303 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3631b77b-c248-4529-9a1d-c5ee04caf3f6 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:47.694899 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0ce301-58b3-4134-b36f-f32b309e8cfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:47.695590 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0ce301-58b3-4134-b36f-f32b309e8cfa admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:47.695899 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0ce301-58b3-4134-b36f-f32b309e8cfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:47.696172 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0ce301-58b3-4134-b36f-f32b309e8cfa admin admin] PATH_INFO: `/v3/projects/2e86ae4c49754b2b920aaa3ab435a08b/users/85801bca7e6f41b2af0d57fe7dbdd272/roles/74a40bc819cb42eea7e966a4b2a016f1` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:47.711151 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0ce301-58b3-4134-b36f-f32b309e8cfa admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e86ae4c49754b2b920aaa3ab435a08b, user_id=85801bca7e6f41b2af0d57fe7dbdd272, role_id=74a40bc819cb42eea7e966a4b2a016f1)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:47.714440 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0ce301-58b3-4134-b36f-f32b309e8cfa admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:47.728329 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1006/2005] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:19:47 2022] PUT /identity/v3/projects/2e86ae4c49754b2b920aaa3ab435a08b/users/85801bca7e6f41b2af0d57fe7dbdd272/roles/74a40bc819cb42eea7e966a4b2a016f1 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:19:47.739566 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6def2bba-b122-4f35-9a9d-09a3287794d9 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:47.739885 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6def2bba-b122-4f35-9a9d-09a3287794d9 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:47.740160 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6def2bba-b122-4f35-9a9d-09a3287794d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:47.771046 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-6def2bba-b122-4f35-9a9d-09a3287794d9 None None] MFA Rules not processed for user `85801bca7e6f41b2af0d57fe7dbdd272`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:19:47.772498 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-6def2bba-b122-4f35-9a9d-09a3287794d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:47.802902 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:19:47.839143 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1000/2006] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:19:47 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:19:47.843699 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0ce301-58b3-4134-b36f-f32b309e8cfa admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:47.890688 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:19:47.909645 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0ce301-58b3-4134-b36f-f32b309e8cfa admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:47.914399 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695fd371-55f8-43af-a209-e9c9b9c26acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:47.915448 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695fd371-55f8-43af-a209-e9c9b9c26acb admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:47.915448 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695fd371-55f8-43af-a209-e9c9b9c26acb admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:47.915634 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695fd371-55f8-43af-a209-e9c9b9c26acb admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:47.916176 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695fd371-55f8-43af-a209-e9c9b9c26acb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:47.918630 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695fd371-55f8-43af-a209-e9c9b9c26acb admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:47.948484 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1007/2007] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 19:19:47 2022] POST /identity/v3/projects => generated 387 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:19:47.952496 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6def2bba-b122-4f35-9a9d-09a3287794d9 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:47.964164 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6def2bba-b122-4f35-9a9d-09a3287794d9 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:48.036209 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd271e63-6804-4d70-ae8c-a3325977fc64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:48.036209 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd271e63-6804-4d70-ae8c-a3325977fc64 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:48.036209 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd271e63-6804-4d70-ae8c-a3325977fc64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:48.036209 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd271e63-6804-4d70-ae8c-a3325977fc64 admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:48.036749 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd271e63-6804-4d70-ae8c-a3325977fc64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:48.036749 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd271e63-6804-4d70-ae8c-a3325977fc64 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:48.054868 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1001/2008] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 19:19:47 2022] POST /identity/v3/users => generated 300 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:19:48.058625 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695fd371-55f8-43af-a209-e9c9b9c26acb admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:48.070423 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695fd371-55f8-43af-a209-e9c9b9c26acb admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:48.075843 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beedfbcf-33b7-4ff8-8651-8289a0b47829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:48.076514 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beedfbcf-33b7-4ff8-8651-8289a0b47829 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:48.076834 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beedfbcf-33b7-4ff8-8651-8289a0b47829 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:48.077096 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beedfbcf-33b7-4ff8-8651-8289a0b47829 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:48.077857 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beedfbcf-33b7-4ff8-8651-8289a0b47829 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:48.079523 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beedfbcf-33b7-4ff8-8651-8289a0b47829 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:48.091685 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1008/2009] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:19:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 19:19:48.095541 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd271e63-6804-4d70-ae8c-a3325977fc64 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:48.107190 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd271e63-6804-4d70-ae8c-a3325977fc64 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:48.120729 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32e6054-d817-4868-8ee3-945ee46faced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:48.121562 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32e6054-d817-4868-8ee3-945ee46faced admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:48.121844 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32e6054-d817-4868-8ee3-945ee46faced admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:48.122114 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32e6054-d817-4868-8ee3-945ee46faced admin admin] PATH_INFO: `/v3/projects/6b08db48ed354981b1c32ee33d7ce304/users/8475d416266b47b2a5c55b9158243714/roles/163c30c3b3c246beb849ce176f02d9da` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:48.138245 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32e6054-d817-4868-8ee3-945ee46faced admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b08db48ed354981b1c32ee33d7ce304, user_id=8475d416266b47b2a5c55b9158243714, role_id=163c30c3b3c246beb849ce176f02d9da)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:48.141001 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32e6054-d817-4868-8ee3-945ee46faced admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:48.164649 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1002/2010] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:19:48 2022] PUT /identity/v3/projects/6b08db48ed354981b1c32ee33d7ce304/users/8475d416266b47b2a5c55b9158243714/roles/163c30c3b3c246beb849ce176f02d9da => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 19:19:48.168538 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beedfbcf-33b7-4ff8-8651-8289a0b47829 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:48.179304 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beedfbcf-33b7-4ff8-8651-8289a0b47829 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:48.236875 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fb921c-5599-4889-a34d-125db3f14057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:48.237485 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fb921c-5599-4889-a34d-125db3f14057 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:48.237826 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fb921c-5599-4889-a34d-125db3f14057 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:48.238101 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fb921c-5599-4889-a34d-125db3f14057 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:48.238766 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fb921c-5599-4889-a34d-125db3f14057 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:48.240399 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fb921c-5599-4889-a34d-125db3f14057 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:48.250429 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1009/2011] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:19:48 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:19:48.254580 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32e6054-d817-4868-8ee3-945ee46faced admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:48.264779 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32e6054-d817-4868-8ee3-945ee46faced admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:48.269332 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e961858d-02fc-47ff-a16b-a9f713982fc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:48.269975 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e961858d-02fc-47ff-a16b-a9f713982fc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:48.270250 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e961858d-02fc-47ff-a16b-a9f713982fc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:48.270521 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e961858d-02fc-47ff-a16b-a9f713982fc5 admin admin] PATH_INFO: `/v3/projects/6b08db48ed354981b1c32ee33d7ce304/users/8475d416266b47b2a5c55b9158243714/roles/163c30c3b3c246beb849ce176f02d9da` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:48.272468 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e961858d-02fc-47ff-a16b-a9f713982fc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b08db48ed354981b1c32ee33d7ce304, user_id=8475d416266b47b2a5c55b9158243714, role_id=163c30c3b3c246beb849ce176f02d9da)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:48.275047 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e961858d-02fc-47ff-a16b-a9f713982fc5 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:48.285144 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1003/2012] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:19:48 2022] PUT /identity/v3/projects/6b08db48ed354981b1c32ee33d7ce304/users/8475d416266b47b2a5c55b9158243714/roles/163c30c3b3c246beb849ce176f02d9da => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 19:19:48.295397 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f49438-6d40-4924-bc78-26a6f6f37cdc None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:48.295716 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f49438-6d40-4924-bc78-26a6f6f37cdc None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:48.295998 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f49438-6d40-4924-bc78-26a6f6f37cdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:48.323595 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-51f49438-6d40-4924-bc78-26a6f6f37cdc None None] MFA Rules not processed for user `8475d416266b47b2a5c55b9158243714`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:19:48.324823 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-51f49438-6d40-4924-bc78-26a6f6f37cdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:48.395547 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1010/2013] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:19:48 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:19:48.406820 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e961858d-02fc-47ff-a16b-a9f713982fc5 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:48.408667 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-e961858d-02fc-47ff-a16b-a9f713982fc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:48.466346 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e961858d-02fc-47ff-a16b-a9f713982fc5 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:48.470012 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c650b3-30ef-46fd-9b7f-dcfb8edaaf4e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e79498873d4ed29a0e19bbea9b9214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:48.470616 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c650b3-30ef-46fd-9b7f-dcfb8edaaf4e service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:48.470893 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c650b3-30ef-46fd-9b7f-dcfb8edaaf4e service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:48.471163 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c650b3-30ef-46fd-9b7f-dcfb8edaaf4e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:48.480507 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c650b3-30ef-46fd-9b7f-dcfb8edaaf4e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:48.481926 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c650b3-30ef-46fd-9b7f-dcfb8edaaf4e service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:48.491222 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1004/2014] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:19:48 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:19:48.520021 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f49438-6d40-4924-bc78-26a6f6f37cdc None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:48.561972 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:19:48.583240 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f49438-6d40-4924-bc78-26a6f6f37cdc None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:48.587887 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5add44c6-4a1b-45e6-874e-fad038acddf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:48.588547 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5add44c6-4a1b-45e6-874e-fad038acddf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:48.588826 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5add44c6-4a1b-45e6-874e-fad038acddf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:48.589095 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5add44c6-4a1b-45e6-874e-fad038acddf7 admin admin] PATH_INFO: `/v3/users/85801bca7e6f41b2af0d57fe7dbdd272` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:48.590225 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5add44c6-4a1b-45e6-874e-fad038acddf7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=85801bca7e6f41b2af0d57fe7dbdd272)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:48.592210 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5add44c6-4a1b-45e6-874e-fad038acddf7 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:48.637658 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-5add44c6-4a1b-45e6-874e-fad038acddf7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85801bca7e6f41b2af0d57fe7dbdd272'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:48.638320 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-5add44c6-4a1b-45e6-874e-fad038acddf7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85801bca7e6f41b2af0d57fe7dbdd272'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:48.645555 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-5add44c6-4a1b-45e6-874e-fad038acddf7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85801bca7e6f41b2af0d57fe7dbdd272'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:48.654632 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-5add44c6-4a1b-45e6-874e-fad038acddf7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85801bca7e6f41b2af0d57fe7dbdd272'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:48.669173 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-5add44c6-4a1b-45e6-874e-fad038acddf7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85801bca7e6f41b2af0d57fe7dbdd272'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:48.675607 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1011/2015] 10.222.0.61 () {58 vars in 1224 bytes} [Mon Aug 22 19:19:48 2022] DELETE /identity/v3/users/85801bca7e6f41b2af0d57fe7dbdd272 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:19:48.680096 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09242f4-4665-41b5-9b5b-d1326cf0a305 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:48.680380 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09242f4-4665-41b5-9b5b-d1326cf0a305 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:48.680652 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09242f4-4665-41b5-9b5b-d1326cf0a305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:48.700792 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-d09242f4-4665-41b5-9b5b-d1326cf0a305 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:19:48.702001 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-d09242f4-4665-41b5-9b5b-d1326cf0a305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:48.720652 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:19:48.763256 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1005/2016] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:19:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:19:48.773032 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5add44c6-4a1b-45e6-874e-fad038acddf7 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:48.774445 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-5add44c6-4a1b-45e6-874e-fad038acddf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:48.844241 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5add44c6-4a1b-45e6-874e-fad038acddf7 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:48.847928 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6f8ce1-a297-4deb-91bc-3b4bf3c05dc2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:48.848546 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6f8ce1-a297-4deb-91bc-3b4bf3c05dc2 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:48.848823 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6f8ce1-a297-4deb-91bc-3b4bf3c05dc2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:48.849096 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6f8ce1-a297-4deb-91bc-3b4bf3c05dc2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:48.859338 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6f8ce1-a297-4deb-91bc-3b4bf3c05dc2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:48.860878 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6f8ce1-a297-4deb-91bc-3b4bf3c05dc2 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:48.871368 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1012/2017] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:19:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 19:19:49.207767 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09242f4-4665-41b5-9b5b-d1326cf0a305 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:49.209317 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-d09242f4-4665-41b5-9b5b-d1326cf0a305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:49.223139 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09242f4-4665-41b5-9b5b-d1326cf0a305 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:49.235482 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb4cc32-1748-4c31-ba7d-f5889a937f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:49.236861 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb4cc32-1748-4c31-ba7d-f5889a937f0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:49.237187 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb4cc32-1748-4c31-ba7d-f5889a937f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:49.237529 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb4cc32-1748-4c31-ba7d-f5889a937f0b admin admin] PATH_INFO: `/v3/projects/2e86ae4c49754b2b920aaa3ab435a08b` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:49.239209 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb4cc32-1748-4c31-ba7d-f5889a937f0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e86ae4c49754b2b920aaa3ab435a08b)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:49.241936 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb4cc32-1748-4c31-ba7d-f5889a937f0b admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:49.305338 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-7bb4cc32-1748-4c31-ba7d-f5889a937f0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e86ae4c49754b2b920aaa3ab435a08b', 'request_id': 'req-7bb4cc32-1748-4c31-ba7d-f5889a937f0b'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:49.315712 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-7bb4cc32-1748-4c31-ba7d-f5889a937f0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e86ae4c49754b2b920aaa3ab435a08b', 'request_id': 'req-7bb4cc32-1748-4c31-ba7d-f5889a937f0b'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:49.322732 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-7bb4cc32-1748-4c31-ba7d-f5889a937f0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e86ae4c49754b2b920aaa3ab435a08b', 'request_id': 'req-7bb4cc32-1748-4c31-ba7d-f5889a937f0b'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:49.336748 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-7bb4cc32-1748-4c31-ba7d-f5889a937f0b admin admin] The token cache is being invalidate because project 2e86ae4c49754b2b920aaa3ab435a08b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 19:19:49.337047 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-7bb4cc32-1748-4c31-ba7d-f5889a937f0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:49.337836 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-7bb4cc32-1748-4c31-ba7d-f5889a937f0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:49.339959 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1006/2018] 10.222.0.61 () {58 vars in 1233 bytes} [Mon Aug 22 19:19:49 2022] DELETE /identity/v3/projects/2e86ae4c49754b2b920aaa3ab435a08b => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:19:49.344430 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6f8ce1-a297-4deb-91bc-3b4bf3c05dc2 service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:49.355786 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6f8ce1-a297-4deb-91bc-3b4bf3c05dc2 service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:49.418337 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27374871-8052-4b02-b1b2-817eb96a9e06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:49.418966 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27374871-8052-4b02-b1b2-817eb96a9e06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:49.419239 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27374871-8052-4b02-b1b2-817eb96a9e06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:49.419506 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27374871-8052-4b02-b1b2-817eb96a9e06 admin admin] PATH_INFO: `/v3/users/8475d416266b47b2a5c55b9158243714` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:49.420678 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27374871-8052-4b02-b1b2-817eb96a9e06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8475d416266b47b2a5c55b9158243714)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:49.422623 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27374871-8052-4b02-b1b2-817eb96a9e06 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:49.428832 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:19:49.473761 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-27374871-8052-4b02-b1b2-817eb96a9e06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8475d416266b47b2a5c55b9158243714'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:49.474514 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-27374871-8052-4b02-b1b2-817eb96a9e06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8475d416266b47b2a5c55b9158243714'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:49.482274 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-27374871-8052-4b02-b1b2-817eb96a9e06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8475d416266b47b2a5c55b9158243714'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:49.489038 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-27374871-8052-4b02-b1b2-817eb96a9e06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8475d416266b47b2a5c55b9158243714'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:49.501437 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-27374871-8052-4b02-b1b2-817eb96a9e06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8475d416266b47b2a5c55b9158243714'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:49.507182 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1013/2019] 10.222.0.61 () {58 vars in 1224 bytes} [Mon Aug 22 19:19:49 2022] DELETE /identity/v3/users/8475d416266b47b2a5c55b9158243714 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:19:49.776372 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb4cc32-1748-4c31-ba7d-f5889a937f0b admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:49.778092 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-7bb4cc32-1748-4c31-ba7d-f5889a937f0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:49.793252 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb4cc32-1748-4c31-ba7d-f5889a937f0b admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:49.852263 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f357801-a121-4168-85a8-2fadd3000354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:49.852899 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f357801-a121-4168-85a8-2fadd3000354 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:49.853178 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f357801-a121-4168-85a8-2fadd3000354 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:49.853448 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f357801-a121-4168-85a8-2fadd3000354 admin admin] PATH_INFO: `/v3/projects/6b08db48ed354981b1c32ee33d7ce304` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:49.855106 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f357801-a121-4168-85a8-2fadd3000354 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b08db48ed354981b1c32ee33d7ce304)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:49.857048 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f357801-a121-4168-85a8-2fadd3000354 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:49.910344 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-2f357801-a121-4168-85a8-2fadd3000354 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b08db48ed354981b1c32ee33d7ce304', 'request_id': 'req-2f357801-a121-4168-85a8-2fadd3000354'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:49.912496 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:19:49.926498 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-2f357801-a121-4168-85a8-2fadd3000354 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b08db48ed354981b1c32ee33d7ce304', 'request_id': 'req-2f357801-a121-4168-85a8-2fadd3000354'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:49.934338 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-2f357801-a121-4168-85a8-2fadd3000354 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b08db48ed354981b1c32ee33d7ce304', 'request_id': 'req-2f357801-a121-4168-85a8-2fadd3000354'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:49.947782 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-2f357801-a121-4168-85a8-2fadd3000354 admin admin] The token cache is being invalidate because project 6b08db48ed354981b1c32ee33d7ce304 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 19:19:49.948081 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-2f357801-a121-4168-85a8-2fadd3000354 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:49.949257 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-2f357801-a121-4168-85a8-2fadd3000354 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:49.954233 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1007/2020] 10.222.0.61 () {58 vars in 1233 bytes} [Mon Aug 22 19:19:49 2022] DELETE /identity/v3/projects/6b08db48ed354981b1c32ee33d7ce304 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:19:49.956171 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc5ce58-c8c8-4014-bfc6-388192977906 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:49.956831 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc5ce58-c8c8-4014-bfc6-388192977906 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:49.957115 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc5ce58-c8c8-4014-bfc6-388192977906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:49.996237 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-7dc5ce58-c8c8-4014-bfc6-388192977906 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:19:49.997461 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-7dc5ce58-c8c8-4014-bfc6-388192977906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:50.055421 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1014/2021] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:19:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 19:19:50.064050 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b8e6c8-ea0d-4ca9-9f09-768d42b0f073 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:50.064402 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b8e6c8-ea0d-4ca9-9f09-768d42b0f073 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:50.064763 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b8e6c8-ea0d-4ca9-9f09-768d42b0f073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:50.084130 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-19b8e6c8-ea0d-4ca9-9f09-768d42b0f073 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:19:50.085335 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-19b8e6c8-ea0d-4ca9-9f09-768d42b0f073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:50.093860 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1008/2022] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:19:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:19:50.097943 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc5ce58-c8c8-4014-bfc6-388192977906 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:50.108439 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc5ce58-c8c8-4014-bfc6-388192977906 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:50.112989 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c942e761-8f82-4735-a85b-11cbe644abba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:50.113594 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c942e761-8f82-4735-a85b-11cbe644abba admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:50.113870 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c942e761-8f82-4735-a85b-11cbe644abba admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:50.114140 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c942e761-8f82-4735-a85b-11cbe644abba admin admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:50.114793 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c942e761-8f82-4735-a85b-11cbe644abba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:19:50.115090 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c942e761-8f82-4735-a85b-11cbe644abba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:50.116726 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c942e761-8f82-4735-a85b-11cbe644abba admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:50.128206 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1015/2023] 10.222.0.61 () {58 vars in 1153 bytes} [Mon Aug 22 19:19:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:19:50.132437 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b8e6c8-ea0d-4ca9-9f09-768d42b0f073 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:50.141592 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b8e6c8-ea0d-4ca9-9f09-768d42b0f073 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:50.146519 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db78ec53-fa0f-4321-b1d3-e740a07fad79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:50.147635 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db78ec53-fa0f-4321-b1d3-e740a07fad79 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:50.147635 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db78ec53-fa0f-4321-b1d3-e740a07fad79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:50.147635 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db78ec53-fa0f-4321-b1d3-e740a07fad79 admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:50.148290 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db78ec53-fa0f-4321-b1d3-e740a07fad79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:50.150192 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db78ec53-fa0f-4321-b1d3-e740a07fad79 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:50.177792 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1009/2024] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 19:19:50 2022] POST /identity/v3/projects => generated 367 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:19:50.181855 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c942e761-8f82-4735-a85b-11cbe644abba admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:50.191868 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c942e761-8f82-4735-a85b-11cbe644abba admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:50.246501 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f308e2-923c-41d3-aec2-0801f8cd7adb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:50.247107 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f308e2-923c-41d3-aec2-0801f8cd7adb admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:50.247383 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f308e2-923c-41d3-aec2-0801f8cd7adb admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:50.247677 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f308e2-923c-41d3-aec2-0801f8cd7adb admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:50.248308 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f308e2-923c-41d3-aec2-0801f8cd7adb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:50.250212 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f308e2-923c-41d3-aec2-0801f8cd7adb admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:50.265589 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:19:50.272398 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1016/2025] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 19:19:50 2022] POST /identity/v3/users => generated 284 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:19:50.276970 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db78ec53-fa0f-4321-b1d3-e740a07fad79 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:50.287317 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db78ec53-fa0f-4321-b1d3-e740a07fad79 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:50.292971 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6193ab65-ec96-440f-8343-30acd4df1c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:50.293604 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6193ab65-ec96-440f-8343-30acd4df1c1c admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:50.293885 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6193ab65-ec96-440f-8343-30acd4df1c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:50.294156 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6193ab65-ec96-440f-8343-30acd4df1c1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:50.294827 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6193ab65-ec96-440f-8343-30acd4df1c1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:50.296531 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6193ab65-ec96-440f-8343-30acd4df1c1c admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:50.306900 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1010/2026] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:19:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:19:50.310704 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f308e2-923c-41d3-aec2-0801f8cd7adb admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:50.321219 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f308e2-923c-41d3-aec2-0801f8cd7adb admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:50.326417 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d12d9ff-ca55-4a32-9e82-88d9efd06190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:50.327021 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d12d9ff-ca55-4a32-9e82-88d9efd06190 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:50.327300 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d12d9ff-ca55-4a32-9e82-88d9efd06190 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:50.327572 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d12d9ff-ca55-4a32-9e82-88d9efd06190 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:50.328282 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d12d9ff-ca55-4a32-9e82-88d9efd06190 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:50.329882 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d12d9ff-ca55-4a32-9e82-88d9efd06190 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:50.341258 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1017/2027] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:19:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:19:50.345425 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6193ab65-ec96-440f-8343-30acd4df1c1c admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:50.356122 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6193ab65-ec96-440f-8343-30acd4df1c1c admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:50.360822 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b835d5f-67a3-41e4-88c7-0c80cf6ea5e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:50.361483 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b835d5f-67a3-41e4-88c7-0c80cf6ea5e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:50.361760 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b835d5f-67a3-41e4-88c7-0c80cf6ea5e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:50.362033 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b835d5f-67a3-41e4-88c7-0c80cf6ea5e6 admin admin] PATH_INFO: `/v3/projects/cc9a19ea3d62466493d02a9f96cb154a/users/844bab0866914f27b0972eb8093b6958/roles/74a40bc819cb42eea7e966a4b2a016f1` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:50.380675 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b835d5f-67a3-41e4-88c7-0c80cf6ea5e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc9a19ea3d62466493d02a9f96cb154a, user_id=844bab0866914f27b0972eb8093b6958, role_id=74a40bc819cb42eea7e966a4b2a016f1)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:50.383424 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b835d5f-67a3-41e4-88c7-0c80cf6ea5e6 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:50.402538 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1011/2028] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:19:50 2022] PUT /identity/v3/projects/cc9a19ea3d62466493d02a9f96cb154a/users/844bab0866914f27b0972eb8093b6958/roles/74a40bc819cb42eea7e966a4b2a016f1 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:19:50.412694 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d72b8-56a9-4210-ae99-788a7560b384 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:50.413208 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d72b8-56a9-4210-ae99-788a7560b384 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:50.413208 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d72b8-56a9-4210-ae99-788a7560b384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:50.445924 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-3d1d72b8-56a9-4210-ae99-788a7560b384 None None] MFA Rules not processed for user `844bab0866914f27b0972eb8093b6958`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:19:50.447154 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-3d1d72b8-56a9-4210-ae99-788a7560b384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:50.519328 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1018/2029] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:19:50 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:19:50.523965 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b835d5f-67a3-41e4-88c7-0c80cf6ea5e6 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:50.587800 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b835d5f-67a3-41e4-88c7-0c80cf6ea5e6 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:50.593618 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc263801-efe2-4bd0-b80c-9258da4073e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:50.594235 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc263801-efe2-4bd0-b80c-9258da4073e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:50.594514 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc263801-efe2-4bd0-b80c-9258da4073e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:50.594785 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc263801-efe2-4bd0-b80c-9258da4073e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:50.595475 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc263801-efe2-4bd0-b80c-9258da4073e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:50.597503 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc263801-efe2-4bd0-b80c-9258da4073e9 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:50.623394 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1012/2030] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 19:19:50 2022] POST /identity/v3/projects => generated 367 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 19:19:50.627456 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1d72b8-56a9-4210-ae99-788a7560b384 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:50.638477 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1d72b8-56a9-4210-ae99-788a7560b384 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:50.697286 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b44c6d9-193d-4a7c-ae57-f0d105f95276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:50.698561 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b44c6d9-193d-4a7c-ae57-f0d105f95276 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:50.698855 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b44c6d9-193d-4a7c-ae57-f0d105f95276 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:50.699127 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b44c6d9-193d-4a7c-ae57-f0d105f95276 admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:50.699819 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b44c6d9-193d-4a7c-ae57-f0d105f95276 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:50.701763 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b44c6d9-193d-4a7c-ae57-f0d105f95276 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:50.725850 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1019/2031] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 19:19:50 2022] POST /identity/v3/users => generated 290 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:19:50.730070 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc263801-efe2-4bd0-b80c-9258da4073e9 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:50.740360 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc263801-efe2-4bd0-b80c-9258da4073e9 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:50.745619 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5142f7a-311b-4891-a557-fedbd4f6c52c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:50.746226 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5142f7a-311b-4891-a557-fedbd4f6c52c admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:50.746570 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5142f7a-311b-4891-a557-fedbd4f6c52c admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:50.746843 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5142f7a-311b-4891-a557-fedbd4f6c52c admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:50.747488 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5142f7a-311b-4891-a557-fedbd4f6c52c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:50.749087 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5142f7a-311b-4891-a557-fedbd4f6c52c admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:50.759689 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1013/2032] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:19:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:19:50.764535 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b44c6d9-193d-4a7c-ae57-f0d105f95276 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:50.774696 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b44c6d9-193d-4a7c-ae57-f0d105f95276 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:50.779757 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec02d02-543c-4c53-aecf-bb6bffb29310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:50.780408 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec02d02-543c-4c53-aecf-bb6bffb29310 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:50.780689 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec02d02-543c-4c53-aecf-bb6bffb29310 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:50.780964 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec02d02-543c-4c53-aecf-bb6bffb29310 admin admin] PATH_INFO: `/v3/projects/9cf65c3fd0b643d990edff75349bcae6/users/bec2fba8388f4a039d2d663c01aa04e4/roles/163c30c3b3c246beb849ce176f02d9da` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:50.796684 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec02d02-543c-4c53-aecf-bb6bffb29310 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9cf65c3fd0b643d990edff75349bcae6, user_id=bec2fba8388f4a039d2d663c01aa04e4, role_id=163c30c3b3c246beb849ce176f02d9da)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:50.799321 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec02d02-543c-4c53-aecf-bb6bffb29310 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:50.812286 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1020/2033] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:19:50 2022] PUT /identity/v3/projects/9cf65c3fd0b643d990edff75349bcae6/users/bec2fba8388f4a039d2d663c01aa04e4/roles/163c30c3b3c246beb849ce176f02d9da => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:19:50.816205 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5142f7a-311b-4891-a557-fedbd4f6c52c admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:50.827040 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5142f7a-311b-4891-a557-fedbd4f6c52c admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:50.870790 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:19:50.884525 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04d0b1f-2358-4c9f-a286-17182afc411a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:50.885711 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04d0b1f-2358-4c9f-a286-17182afc411a admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:50.885991 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04d0b1f-2358-4c9f-a286-17182afc411a admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:50.886264 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04d0b1f-2358-4c9f-a286-17182afc411a admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:50.886906 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04d0b1f-2358-4c9f-a286-17182afc411a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:50.888530 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04d0b1f-2358-4c9f-a286-17182afc411a admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:50.898236 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1014/2034] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:19:50 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:19:50.903218 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec02d02-543c-4c53-aecf-bb6bffb29310 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:50.913549 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec02d02-543c-4c53-aecf-bb6bffb29310 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:50.918143 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c144f9-9d45-423c-8e3c-4b6b0bf0c2e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:50.918793 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c144f9-9d45-423c-8e3c-4b6b0bf0c2e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:50.919067 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c144f9-9d45-423c-8e3c-4b6b0bf0c2e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:50.919338 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c144f9-9d45-423c-8e3c-4b6b0bf0c2e5 admin admin] PATH_INFO: `/v3/projects/9cf65c3fd0b643d990edff75349bcae6/users/bec2fba8388f4a039d2d663c01aa04e4/roles/163c30c3b3c246beb849ce176f02d9da` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:50.921285 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c144f9-9d45-423c-8e3c-4b6b0bf0c2e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9cf65c3fd0b643d990edff75349bcae6, user_id=bec2fba8388f4a039d2d663c01aa04e4, role_id=163c30c3b3c246beb849ce176f02d9da)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:50.923926 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c144f9-9d45-423c-8e3c-4b6b0bf0c2e5 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:50.934358 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1021/2035] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:19:50 2022] PUT /identity/v3/projects/9cf65c3fd0b643d990edff75349bcae6/users/bec2fba8388f4a039d2d663c01aa04e4/roles/163c30c3b3c246beb849ce176f02d9da => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:19:50.944648 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a61b5c-dd39-4cd0-aecb-00b4b272e29b None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:50.944934 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a61b5c-dd39-4cd0-aecb-00b4b272e29b None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:50.945208 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a61b5c-dd39-4cd0-aecb-00b4b272e29b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:50.973519 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-75a61b5c-dd39-4cd0-aecb-00b4b272e29b None None] MFA Rules not processed for user `bec2fba8388f4a039d2d663c01aa04e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:19:50.974733 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-75a61b5c-dd39-4cd0-aecb-00b4b272e29b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:51.078452 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1015/2036] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:19:50 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:19:51.092663 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c144f9-9d45-423c-8e3c-4b6b0bf0c2e5 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:51.095028 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-31c144f9-9d45-423c-8e3c-4b6b0bf0c2e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:51.141134 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:19:51.174217 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c144f9-9d45-423c-8e3c-4b6b0bf0c2e5 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:51.178768 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9a88f8-d8a3-400f-a254-a630be56b236 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e79498873d4ed29a0e19bbea9b9214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:51.179576 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9a88f8-d8a3-400f-a254-a630be56b236 service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:51.180219 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9a88f8-d8a3-400f-a254-a630be56b236 service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:51.180219 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9a88f8-d8a3-400f-a254-a630be56b236 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:51.191153 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9a88f8-d8a3-400f-a254-a630be56b236 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:51.192642 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9a88f8-d8a3-400f-a254-a630be56b236 service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:51.204697 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1022/2037] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:19:51 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 19:19:51.295614 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a61b5c-dd39-4cd0-aecb-00b4b272e29b None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:51.306539 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a61b5c-dd39-4cd0-aecb-00b4b272e29b None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:51.310292 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5a1249-542d-4246-870c-d8c369218200 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e79498873d4ed29a0e19bbea9b9214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:51.310893 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5a1249-542d-4246-870c-d8c369218200 service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:51.311165 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5a1249-542d-4246-870c-d8c369218200 service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:51.311432 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5a1249-542d-4246-870c-d8c369218200 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:51.372794 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5a1249-542d-4246-870c-d8c369218200 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:51.374261 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5a1249-542d-4246-870c-d8c369218200 service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:51.383422 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1016/2038] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:19:51 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 19:19:52.960545 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9a88f8-d8a3-400f-a254-a630be56b236 service cinder] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:52.960545 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-ad9a88f8-d8a3-400f-a254-a630be56b236 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:53.058107 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9a88f8-d8a3-400f-a254-a630be56b236 service cinder] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:53.063250 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c31112-4ac6-4f0d-95fa-6650f809f12b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:53.064026 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c31112-4ac6-4f0d-95fa-6650f809f12b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:53.064804 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c31112-4ac6-4f0d-95fa-6650f809f12b admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:53.064804 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c31112-4ac6-4f0d-95fa-6650f809f12b admin admin] PATH_INFO: `/v3/users/3bfd09479ca64c3f86d0d73763e29464` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:53.067383 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c31112-4ac6-4f0d-95fa-6650f809f12b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3bfd09479ca64c3f86d0d73763e29464)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:53.069608 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c31112-4ac6-4f0d-95fa-6650f809f12b admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:53.162792 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-b8c31112-4ac6-4f0d-95fa-6650f809f12b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3bfd09479ca64c3f86d0d73763e29464'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:53.163775 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-b8c31112-4ac6-4f0d-95fa-6650f809f12b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3bfd09479ca64c3f86d0d73763e29464'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:53.173332 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-b8c31112-4ac6-4f0d-95fa-6650f809f12b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3bfd09479ca64c3f86d0d73763e29464'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:53.184451 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-b8c31112-4ac6-4f0d-95fa-6650f809f12b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3bfd09479ca64c3f86d0d73763e29464'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:53.209145 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-b8c31112-4ac6-4f0d-95fa-6650f809f12b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3bfd09479ca64c3f86d0d73763e29464'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:53.219694 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1023/2039] 10.222.0.61 () {58 vars in 1224 bytes} [Mon Aug 22 19:19:52 2022] DELETE /identity/v3/users/3bfd09479ca64c3f86d0d73763e29464 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:19:53.224735 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547d6dca-f97c-47f3-8b9e-4b290085bdab None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:53.225246 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547d6dca-f97c-47f3-8b9e-4b290085bdab None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:53.225848 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547d6dca-f97c-47f3-8b9e-4b290085bdab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:53.249701 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-547d6dca-f97c-47f3-8b9e-4b290085bdab None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:19:53.251076 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-547d6dca-f97c-47f3-8b9e-4b290085bdab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:53.357477 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1017/2040] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:19:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 19:19:53.367250 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c31112-4ac6-4f0d-95fa-6650f809f12b admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:53.368719 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-b8c31112-4ac6-4f0d-95fa-6650f809f12b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:53.437368 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c31112-4ac6-4f0d-95fa-6650f809f12b admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:53.442621 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d1e402-7272-4e76-b150-da2f96a5d206 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:53.443445 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d1e402-7272-4e76-b150-da2f96a5d206 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:53.443921 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d1e402-7272-4e76-b150-da2f96a5d206 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:53.444348 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d1e402-7272-4e76-b150-da2f96a5d206 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:53.452108 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:19:53.456647 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d1e402-7272-4e76-b150-da2f96a5d206 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:53.458266 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d1e402-7272-4e76-b150-da2f96a5d206 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:53.475030 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1024/2041] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:19:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 19:19:53.840249 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547d6dca-f97c-47f3-8b9e-4b290085bdab None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:53.842149 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-547d6dca-f97c-47f3-8b9e-4b290085bdab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:53.864718 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547d6dca-f97c-47f3-8b9e-4b290085bdab None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:53.870680 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fed988-3552-4f69-90e9-51678f4b5085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:53.871323 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fed988-3552-4f69-90e9-51678f4b5085 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:53.871598 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fed988-3552-4f69-90e9-51678f4b5085 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:53.871900 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fed988-3552-4f69-90e9-51678f4b5085 admin admin] PATH_INFO: `/v3/projects/a15343703c7144bf8677126a9ada39da` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:53.873204 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fed988-3552-4f69-90e9-51678f4b5085 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a15343703c7144bf8677126a9ada39da)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:53.876006 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fed988-3552-4f69-90e9-51678f4b5085 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:53.878966 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:19:53.931436 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a5fed988-3552-4f69-90e9-51678f4b5085 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a15343703c7144bf8677126a9ada39da', 'request_id': 'req-a5fed988-3552-4f69-90e9-51678f4b5085'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:53.939431 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a5fed988-3552-4f69-90e9-51678f4b5085 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a15343703c7144bf8677126a9ada39da', 'request_id': 'req-a5fed988-3552-4f69-90e9-51678f4b5085'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:53.946265 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a5fed988-3552-4f69-90e9-51678f4b5085 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a15343703c7144bf8677126a9ada39da', 'request_id': 'req-a5fed988-3552-4f69-90e9-51678f4b5085'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:53.958892 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a5fed988-3552-4f69-90e9-51678f4b5085 admin admin] The token cache is being invalidate because project a15343703c7144bf8677126a9ada39da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 19:19:53.959183 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a5fed988-3552-4f69-90e9-51678f4b5085 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:53.960832 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a5fed988-3552-4f69-90e9-51678f4b5085 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:53.961431 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1018/2042] 10.222.0.61 () {58 vars in 1233 bytes} [Mon Aug 22 19:19:53 2022] DELETE /identity/v3/projects/a15343703c7144bf8677126a9ada39da => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:19:53.966182 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d1e402-7272-4e76-b150-da2f96a5d206 service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:53.973866 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fed988-3552-4f69-90e9-51678f4b5085 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:53.975266 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-a5fed988-3552-4f69-90e9-51678f4b5085 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:53.979558 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d1e402-7272-4e76-b150-da2f96a5d206 service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:54.060321 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fed988-3552-4f69-90e9-51678f4b5085 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:54.062612 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e1f152-6b58-44c3-af68-b86d568ec84d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:54.063623 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e1f152-6b58-44c3-af68-b86d568ec84d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:54.064073 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e1f152-6b58-44c3-af68-b86d568ec84d admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:54.064493 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e1f152-6b58-44c3-af68-b86d568ec84d admin admin] PATH_INFO: `/v3/users/8b85da6d1d414ff3ae823a8e307b2321` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:54.066020 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e1f152-6b58-44c3-af68-b86d568ec84d admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b85da6d1d414ff3ae823a8e307b2321)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:54.067194 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c8456a-27f3-4517-be54-d38674a17551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:54.068089 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c8456a-27f3-4517-be54-d38674a17551 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:54.068481 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c8456a-27f3-4517-be54-d38674a17551 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:54.068855 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c8456a-27f3-4517-be54-d38674a17551 admin admin] PATH_INFO: `/v3/users/13452ef446614d13a81f83f101cb69b8` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:54.069316 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e1f152-6b58-44c3-af68-b86d568ec84d admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:54.070235 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c8456a-27f3-4517-be54-d38674a17551 admin admin] RBAC: Authorizing `identity:delete_user(user_id=13452ef446614d13a81f83f101cb69b8)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:54.073301 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c8456a-27f3-4517-be54-d38674a17551 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:54.171174 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-37e1f152-6b58-44c3-af68-b86d568ec84d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b85da6d1d414ff3ae823a8e307b2321'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:54.172169 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-37e1f152-6b58-44c3-af68-b86d568ec84d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b85da6d1d414ff3ae823a8e307b2321'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:54.175340 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a5c8456a-27f3-4517-be54-d38674a17551 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13452ef446614d13a81f83f101cb69b8'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:54.176478 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a5c8456a-27f3-4517-be54-d38674a17551 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13452ef446614d13a81f83f101cb69b8'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:54.184170 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-37e1f152-6b58-44c3-af68-b86d568ec84d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b85da6d1d414ff3ae823a8e307b2321'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:54.187440 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a5c8456a-27f3-4517-be54-d38674a17551 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13452ef446614d13a81f83f101cb69b8'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:54.195666 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-37e1f152-6b58-44c3-af68-b86d568ec84d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b85da6d1d414ff3ae823a8e307b2321'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:54.199368 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a5c8456a-27f3-4517-be54-d38674a17551 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13452ef446614d13a81f83f101cb69b8'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:54.216984 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-37e1f152-6b58-44c3-af68-b86d568ec84d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b85da6d1d414ff3ae823a8e307b2321'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:54.220923 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a5c8456a-27f3-4517-be54-d38674a17551 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13452ef446614d13a81f83f101cb69b8'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:54.230102 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1019/2043] 10.222.0.61 () {58 vars in 1224 bytes} [Mon Aug 22 19:19:53 2022] DELETE /identity/v3/users/13452ef446614d13a81f83f101cb69b8 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 19:19:54.237228 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a62440-47cb-498b-a398-752f75270a1f None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:54.237228 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a62440-47cb-498b-a398-752f75270a1f None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:54.237228 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a62440-47cb-498b-a398-752f75270a1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:54.238423 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1025/2044] 10.222.0.61 () {58 vars in 1224 bytes} [Mon Aug 22 19:19:53 2022] DELETE /identity/v3/users/8b85da6d1d414ff3ae823a8e307b2321 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:19:54.261343 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-21a62440-47cb-498b-a398-752f75270a1f None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:19:54.263489 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-21a62440-47cb-498b-a398-752f75270a1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:54.271138 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:19:54.366959 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1020/2045] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:19:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 19:19:54.384297 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e1f152-6b58-44c3-af68-b86d568ec84d admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:54.385879 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-37e1f152-6b58-44c3-af68-b86d568ec84d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:54.524961 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e1f152-6b58-44c3-af68-b86d568ec84d admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:54.548040 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8625add9-abfe-4a56-b43b-50fabf24e7aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:54.549199 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8625add9-abfe-4a56-b43b-50fabf24e7aa service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:54.549199 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8625add9-abfe-4a56-b43b-50fabf24e7aa service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:54.549555 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8625add9-abfe-4a56-b43b-50fabf24e7aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:54.562136 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8625add9-abfe-4a56-b43b-50fabf24e7aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:54.563782 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8625add9-abfe-4a56-b43b-50fabf24e7aa service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:54.609881 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1026/2046] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:19:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:19:54.946675 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a62440-47cb-498b-a398-752f75270a1f None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:54.948822 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-21a62440-47cb-498b-a398-752f75270a1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:54.976346 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a62440-47cb-498b-a398-752f75270a1f None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:54.988870 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688d6341-ebcb-4143-97eb-2cc385b3b0c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:54.989555 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688d6341-ebcb-4143-97eb-2cc385b3b0c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:54.989905 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688d6341-ebcb-4143-97eb-2cc385b3b0c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:54.990183 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688d6341-ebcb-4143-97eb-2cc385b3b0c6 admin admin] PATH_INFO: `/v3/projects/dd36ddc018e34297844281df4a35239c` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:54.994821 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688d6341-ebcb-4143-97eb-2cc385b3b0c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd36ddc018e34297844281df4a35239c)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:54.996811 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688d6341-ebcb-4143-97eb-2cc385b3b0c6 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:55.091493 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-688d6341-ebcb-4143-97eb-2cc385b3b0c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd36ddc018e34297844281df4a35239c', 'request_id': 'req-688d6341-ebcb-4143-97eb-2cc385b3b0c6'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:55.105061 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-688d6341-ebcb-4143-97eb-2cc385b3b0c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd36ddc018e34297844281df4a35239c', 'request_id': 'req-688d6341-ebcb-4143-97eb-2cc385b3b0c6'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:55.129946 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-688d6341-ebcb-4143-97eb-2cc385b3b0c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd36ddc018e34297844281df4a35239c', 'request_id': 'req-688d6341-ebcb-4143-97eb-2cc385b3b0c6'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:55.145299 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-688d6341-ebcb-4143-97eb-2cc385b3b0c6 admin admin] The token cache is being invalidate because project dd36ddc018e34297844281df4a35239c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 19:19:55.145524 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-688d6341-ebcb-4143-97eb-2cc385b3b0c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:55.146188 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-688d6341-ebcb-4143-97eb-2cc385b3b0c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:55.147742 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1021/2047] 10.222.0.61 () {58 vars in 1233 bytes} [Mon Aug 22 19:19:54 2022] DELETE /identity/v3/projects/dd36ddc018e34297844281df4a35239c => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:19:55.154662 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e84aaa-7410-444b-8c18-eea98df394cc None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:55.155531 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e84aaa-7410-444b-8c18-eea98df394cc None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:55.156092 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e84aaa-7410-444b-8c18-eea98df394cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:55.179175 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-40e84aaa-7410-444b-8c18-eea98df394cc None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:19:55.180653 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-40e84aaa-7410-444b-8c18-eea98df394cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:55.194012 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:19:55.265317 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688d6341-ebcb-4143-97eb-2cc385b3b0c6 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:55.266820 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-688d6341-ebcb-4143-97eb-2cc385b3b0c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:55.301447 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1027/2048] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:19:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:19:55.311952 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82c1203-b0f2-4c11-aad1-b4a116d9fae8 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:55.312229 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82c1203-b0f2-4c11-aad1-b4a116d9fae8 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:55.313485 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82c1203-b0f2-4c11-aad1-b4a116d9fae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:55.356393 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-b82c1203-b0f2-4c11-aad1-b4a116d9fae8 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:19:55.359222 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-b82c1203-b0f2-4c11-aad1-b4a116d9fae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:55.379512 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688d6341-ebcb-4143-97eb-2cc385b3b0c6 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:55.379936 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1028/2049] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:19:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:19:55.385158 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82c1203-b0f2-4c11-aad1-b4a116d9fae8 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:55.404836 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782a2be7-e5ab-4bcc-bab9-0d6900754f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:55.406923 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782a2be7-e5ab-4bcc-bab9-0d6900754f93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:55.408421 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782a2be7-e5ab-4bcc-bab9-0d6900754f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:55.408866 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782a2be7-e5ab-4bcc-bab9-0d6900754f93 admin admin] PATH_INFO: `/v3/projects/cd7e34cfc21b4b9c9ccc992faaadb75f` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:55.411148 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782a2be7-e5ab-4bcc-bab9-0d6900754f93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd7e34cfc21b4b9c9ccc992faaadb75f)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:55.416024 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782a2be7-e5ab-4bcc-bab9-0d6900754f93 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:55.426012 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82c1203-b0f2-4c11-aad1-b4a116d9fae8 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:55.443002 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1191244c-4693-40a0-98bf-a794d3e67214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:55.446314 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1191244c-4693-40a0-98bf-a794d3e67214 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:55.447994 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1191244c-4693-40a0-98bf-a794d3e67214 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:55.448392 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1191244c-4693-40a0-98bf-a794d3e67214 admin admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:55.450405 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1191244c-4693-40a0-98bf-a794d3e67214 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:19:55.452225 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1191244c-4693-40a0-98bf-a794d3e67214 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:55.457471 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1191244c-4693-40a0-98bf-a794d3e67214 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:55.495371 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1029/2050] 10.222.0.61 () {58 vars in 1153 bytes} [Mon Aug 22 19:19:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:19:55.501965 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1191244c-4693-40a0-98bf-a794d3e67214 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:55.522079 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:19:55.529007 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1191244c-4693-40a0-98bf-a794d3e67214 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:55.558725 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-782a2be7-e5ab-4bcc-bab9-0d6900754f93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd7e34cfc21b4b9c9ccc992faaadb75f', 'request_id': 'req-782a2be7-e5ab-4bcc-bab9-0d6900754f93'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:55.576347 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-782a2be7-e5ab-4bcc-bab9-0d6900754f93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd7e34cfc21b4b9c9ccc992faaadb75f', 'request_id': 'req-782a2be7-e5ab-4bcc-bab9-0d6900754f93'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:55.596815 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-782a2be7-e5ab-4bcc-bab9-0d6900754f93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd7e34cfc21b4b9c9ccc992faaadb75f', 'request_id': 'req-782a2be7-e5ab-4bcc-bab9-0d6900754f93'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:55.620467 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa4a50f-20a2-4702-9b2a-92e48522c00f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:55.621270 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa4a50f-20a2-4702-9b2a-92e48522c00f admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:55.622432 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa4a50f-20a2-4702-9b2a-92e48522c00f admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:55.622802 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa4a50f-20a2-4702-9b2a-92e48522c00f admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:55.623671 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa4a50f-20a2-4702-9b2a-92e48522c00f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:55.625896 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa4a50f-20a2-4702-9b2a-92e48522c00f admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:55.636424 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-782a2be7-e5ab-4bcc-bab9-0d6900754f93 admin admin] The token cache is being invalidate because project cd7e34cfc21b4b9c9ccc992faaadb75f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 19:19:55.636716 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-782a2be7-e5ab-4bcc-bab9-0d6900754f93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:55.640166 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-782a2be7-e5ab-4bcc-bab9-0d6900754f93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:55.648277 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1022/2051] 10.222.0.61 () {58 vars in 1233 bytes} [Mon Aug 22 19:19:55 2022] DELETE /identity/v3/projects/cd7e34cfc21b4b9c9ccc992faaadb75f => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 19:19:55.655332 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc1241d-de57-4869-a334-ce952cc4e517 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:55.656154 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc1241d-de57-4869-a334-ce952cc4e517 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:55.656671 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc1241d-de57-4869-a334-ce952cc4e517 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:55.689839 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-3fc1241d-de57-4869-a334-ce952cc4e517 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:19:55.693203 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-3fc1241d-de57-4869-a334-ce952cc4e517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:55.710270 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1030/2052] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 19:19:55 2022] POST /identity/v3/projects => generated 385 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:19:55.712267 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa4a50f-20a2-4702-9b2a-92e48522c00f admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:55.718321 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-4fa4a50f-20a2-4702-9b2a-92e48522c00f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:55.743050 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1023/2053] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:19:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:19:55.749089 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21231080-313e-4ca3-8658-13040b95e82a None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:55.749654 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21231080-313e-4ca3-8658-13040b95e82a None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:55.750056 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21231080-313e-4ca3-8658-13040b95e82a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:55.813044 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-21231080-313e-4ca3-8658-13040b95e82a None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:19:55.816355 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-21231080-313e-4ca3-8658-13040b95e82a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:55.887522 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa4a50f-20a2-4702-9b2a-92e48522c00f admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:55.896914 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1024/2054] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:19:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:19:55.897770 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be37d362-433d-4a79-adfe-33e8bbff8766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:55.898405 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be37d362-433d-4a79-adfe-33e8bbff8766 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:55.898683 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be37d362-433d-4a79-adfe-33e8bbff8766 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:55.899006 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be37d362-433d-4a79-adfe-33e8bbff8766 admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:55.902386 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21231080-313e-4ca3-8658-13040b95e82a None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:55.905589 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be37d362-433d-4a79-adfe-33e8bbff8766 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:55.907570 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be37d362-433d-4a79-adfe-33e8bbff8766 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:55.917978 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21231080-313e-4ca3-8658-13040b95e82a None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:55.938380 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72522b28-a64e-4a48-86ef-7eb626f821c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:55.938970 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72522b28-a64e-4a48-86ef-7eb626f821c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:55.939250 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72522b28-a64e-4a48-86ef-7eb626f821c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:55.939523 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72522b28-a64e-4a48-86ef-7eb626f821c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:55.940223 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72522b28-a64e-4a48-86ef-7eb626f821c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:19:55.940525 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72522b28-a64e-4a48-86ef-7eb626f821c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:55.944159 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1031/2055] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 19:19:55 2022] POST /identity/v3/users => generated 293 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:19:55.944285 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72522b28-a64e-4a48-86ef-7eb626f821c3 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:55.945518 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be37d362-433d-4a79-adfe-33e8bbff8766 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:55.958621 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be37d362-433d-4a79-adfe-33e8bbff8766 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:55.959148 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1025/2056] 10.222.0.61 () {58 vars in 1153 bytes} [Mon Aug 22 19:19:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:19:55.963860 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72522b28-a64e-4a48-86ef-7eb626f821c3 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:55.965643 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1747cca-ead7-4ced-84d5-0c121e74f905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:55.966668 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1747cca-ead7-4ced-84d5-0c121e74f905 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:55.966668 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1747cca-ead7-4ced-84d5-0c121e74f905 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:55.967004 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1747cca-ead7-4ced-84d5-0c121e74f905 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:55.967687 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1747cca-ead7-4ced-84d5-0c121e74f905 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:55.969764 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1747cca-ead7-4ced-84d5-0c121e74f905 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:55.973917 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72522b28-a64e-4a48-86ef-7eb626f821c3 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:55.975409 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:19:55.982292 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249bc5a9-1a23-46cc-9170-1d231322272f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:55.982601 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1032/2057] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:19:55 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:19:55.983050 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249bc5a9-1a23-46cc-9170-1d231322272f admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:55.983745 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249bc5a9-1a23-46cc-9170-1d231322272f admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:55.983856 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249bc5a9-1a23-46cc-9170-1d231322272f admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:55.984650 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249bc5a9-1a23-46cc-9170-1d231322272f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:55.986820 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1747cca-ead7-4ced-84d5-0c121e74f905 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:55.987396 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249bc5a9-1a23-46cc-9170-1d231322272f admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:56.002091 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1747cca-ead7-4ced-84d5-0c121e74f905 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:56.014703 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9ab85f-1521-4e92-a6be-f194c68abe28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:56.017065 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9ab85f-1521-4e92-a6be-f194c68abe28 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:56.017640 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9ab85f-1521-4e92-a6be-f194c68abe28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:56.017945 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9ab85f-1521-4e92-a6be-f194c68abe28 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:56.019260 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9ab85f-1521-4e92-a6be-f194c68abe28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:56.021580 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1026/2058] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 19:19:55 2022] POST /identity/v3/projects => generated 375 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 19:19:56.024767 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9ab85f-1521-4e92-a6be-f194c68abe28 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:56.026324 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249bc5a9-1a23-46cc-9170-1d231322272f admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:56.042847 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1033/2059] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:19:55 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:19:56.047091 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9ab85f-1521-4e92-a6be-f194c68abe28 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:56.065205 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9ab85f-1521-4e92-a6be-f194c68abe28 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:56.123358 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249bc5a9-1a23-46cc-9170-1d231322272f admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:56.129932 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a6a600-985e-4738-9d42-69993fb4ed69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:56.131221 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a6a600-985e-4738-9d42-69993fb4ed69 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:56.131723 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a6a600-985e-4738-9d42-69993fb4ed69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:56.132148 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a6a600-985e-4738-9d42-69993fb4ed69 admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:56.133997 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a6a600-985e-4738-9d42-69993fb4ed69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:56.136095 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a6a600-985e-4738-9d42-69993fb4ed69 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:56.162008 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167cd11a-ce71-4fec-a61f-444a4c60cc1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:56.163623 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167cd11a-ce71-4fec-a61f-444a4c60cc1b admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:56.164308 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167cd11a-ce71-4fec-a61f-444a4c60cc1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:56.164816 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167cd11a-ce71-4fec-a61f-444a4c60cc1b admin admin] PATH_INFO: `/v3/projects/8ce7437988394d0aa37f4a7bb3ecba39/users/1df1dfa42d674ea8bd1c3c940a2d8b5b/roles/74a40bc819cb42eea7e966a4b2a016f1` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:56.177000 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1027/2060] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 19:19:56 2022] POST /identity/v3/users => generated 288 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:19:56.178873 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a6a600-985e-4738-9d42-69993fb4ed69 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:56.182591 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167cd11a-ce71-4fec-a61f-444a4c60cc1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ce7437988394d0aa37f4a7bb3ecba39, user_id=1df1dfa42d674ea8bd1c3c940a2d8b5b, role_id=74a40bc819cb42eea7e966a4b2a016f1)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:56.185806 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167cd11a-ce71-4fec-a61f-444a4c60cc1b admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:56.191764 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a6a600-985e-4738-9d42-69993fb4ed69 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:56.197433 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ef4203-e583-4fcc-9d01-b38669177349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:56.198264 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ef4203-e583-4fcc-9d01-b38669177349 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:56.198541 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ef4203-e583-4fcc-9d01-b38669177349 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:56.198808 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ef4203-e583-4fcc-9d01-b38669177349 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:56.199507 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ef4203-e583-4fcc-9d01-b38669177349 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:56.201256 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ef4203-e583-4fcc-9d01-b38669177349 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:56.213543 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1028/2061] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:19:56 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:19:56.217378 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ef4203-e583-4fcc-9d01-b38669177349 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:56.222691 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1034/2062] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:19:56 2022] PUT /identity/v3/projects/8ce7437988394d0aa37f4a7bb3ecba39/users/1df1dfa42d674ea8bd1c3c940a2d8b5b/roles/74a40bc819cb42eea7e966a4b2a016f1 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:19:56.223377 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:19:56.235397 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6478ed1e-a2bd-442c-8df6-5a07794ff392 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:56.235717 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6478ed1e-a2bd-442c-8df6-5a07794ff392 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:56.235995 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6478ed1e-a2bd-442c-8df6-5a07794ff392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:56.239134 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ef4203-e583-4fcc-9d01-b38669177349 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:56.269725 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-6478ed1e-a2bd-442c-8df6-5a07794ff392 None None] MFA Rules not processed for user `1df1dfa42d674ea8bd1c3c940a2d8b5b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:19:56.270949 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-6478ed1e-a2bd-442c-8df6-5a07794ff392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:56.308292 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d119217e-b4e9-4df6-8df5-8476883dd2b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:56.308910 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d119217e-b4e9-4df6-8df5-8476883dd2b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:56.309174 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d119217e-b4e9-4df6-8df5-8476883dd2b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:56.309450 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d119217e-b4e9-4df6-8df5-8476883dd2b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:56.310103 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d119217e-b4e9-4df6-8df5-8476883dd2b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:56.312450 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d119217e-b4e9-4df6-8df5-8476883dd2b4 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:56.326058 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1029/2063] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:19:56 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:19:56.330296 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d119217e-b4e9-4df6-8df5-8476883dd2b4 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:56.343319 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d119217e-b4e9-4df6-8df5-8476883dd2b4 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:56.349022 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3eaa89-ff03-421c-b8bb-d76c6ea2576f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:56.350023 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3eaa89-ff03-421c-b8bb-d76c6ea2576f admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:56.350404 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3eaa89-ff03-421c-b8bb-d76c6ea2576f admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:56.350780 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3eaa89-ff03-421c-b8bb-d76c6ea2576f admin admin] PATH_INFO: `/v3/projects/228376dec9af4deabeef24c9b801a300/users/a5a9dd005a9d44f79d298ccc70d38bac/roles/74a40bc819cb42eea7e966a4b2a016f1` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:56.360195 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1035/2064] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:19:56 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:19:56.376630 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3eaa89-ff03-421c-b8bb-d76c6ea2576f admin admin] RBAC: Authorizing `identity:create_grant(project_id=228376dec9af4deabeef24c9b801a300, user_id=a5a9dd005a9d44f79d298ccc70d38bac, role_id=74a40bc819cb42eea7e966a4b2a016f1)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:56.379690 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3eaa89-ff03-421c-b8bb-d76c6ea2576f admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:56.381370 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6478ed1e-a2bd-442c-8df6-5a07794ff392 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:56.383034 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-6478ed1e-a2bd-442c-8df6-5a07794ff392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:56.403011 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1030/2065] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:19:56 2022] PUT /identity/v3/projects/228376dec9af4deabeef24c9b801a300/users/a5a9dd005a9d44f79d298ccc70d38bac/roles/74a40bc819cb42eea7e966a4b2a016f1 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:19:56.414348 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6eb6e0-6413-40d9-9cf2-edaa9839992a None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:56.414809 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6eb6e0-6413-40d9-9cf2-edaa9839992a None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:56.415183 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6eb6e0-6413-40d9-9cf2-edaa9839992a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:56.459864 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-5b6eb6e0-6413-40d9-9cf2-edaa9839992a None None] MFA Rules not processed for user `a5a9dd005a9d44f79d298ccc70d38bac`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:19:56.461278 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-5b6eb6e0-6413-40d9-9cf2-edaa9839992a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:56.481767 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6478ed1e-a2bd-442c-8df6-5a07794ff392 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:56.494735 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:19:56.535235 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de05f0c-9e5c-479b-bd0b-590cd461a917 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e79498873d4ed29a0e19bbea9b9214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:56.536031 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de05f0c-9e5c-479b-bd0b-590cd461a917 service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:56.536418 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de05f0c-9e5c-479b-bd0b-590cd461a917 service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:56.536784 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de05f0c-9e5c-479b-bd0b-590cd461a917 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:56.539495 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1031/2066] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:19:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 19:19:56.544363 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6eb6e0-6413-40d9-9cf2-edaa9839992a None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:56.611455 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de05f0c-9e5c-479b-bd0b-590cd461a917 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:56.613059 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de05f0c-9e5c-479b-bd0b-590cd461a917 service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:56.623296 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1036/2067] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:19:56 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 19:19:56.645823 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de05f0c-9e5c-479b-bd0b-590cd461a917 service cinder] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:56.648312 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6eb6e0-6413-40d9-9cf2-edaa9839992a None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:56.655550 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae2a0d6-48be-41c0-a379-9d1c81de3be3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:56.656328 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae2a0d6-48be-41c0-a379-9d1c81de3be3 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:56.656698 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae2a0d6-48be-41c0-a379-9d1c81de3be3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:56.657066 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae2a0d6-48be-41c0-a379-9d1c81de3be3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:56.657801 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae2a0d6-48be-41c0-a379-9d1c81de3be3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:56.659962 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae2a0d6-48be-41c0-a379-9d1c81de3be3 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:56.666962 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de05f0c-9e5c-479b-bd0b-590cd461a917 service cinder] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:56.671131 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:19:56.672614 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af58a87-27e3-4167-aa1f-8467eb3b87be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:56.673259 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af58a87-27e3-4167-aa1f-8467eb3b87be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:56.673659 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af58a87-27e3-4167-aa1f-8467eb3b87be admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:56.673797 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af58a87-27e3-4167-aa1f-8467eb3b87be admin admin] PATH_INFO: `/v3/users/1df1dfa42d674ea8bd1c3c940a2d8b5b` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:56.677344 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af58a87-27e3-4167-aa1f-8467eb3b87be admin admin] RBAC: Authorizing `identity:delete_user(user_id=1df1dfa42d674ea8bd1c3c940a2d8b5b)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:56.678125 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af58a87-27e3-4167-aa1f-8467eb3b87be admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:56.703534 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1032/2068] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 19:19:56 2022] POST /identity/v3/projects => generated 377 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 19:19:56.704632 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae2a0d6-48be-41c0-a379-9d1c81de3be3 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:56.787237 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-5af58a87-27e3-4167-aa1f-8467eb3b87be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1df1dfa42d674ea8bd1c3c940a2d8b5b'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:56.789599 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-5af58a87-27e3-4167-aa1f-8467eb3b87be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1df1dfa42d674ea8bd1c3c940a2d8b5b'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:56.800541 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae2a0d6-48be-41c0-a379-9d1c81de3be3 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:56.802728 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-5af58a87-27e3-4167-aa1f-8467eb3b87be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1df1dfa42d674ea8bd1c3c940a2d8b5b'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:56.807395 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fae2bf-e372-4a45-934f-9169e60dc444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:56.808227 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fae2bf-e372-4a45-934f-9169e60dc444 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:56.808554 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fae2bf-e372-4a45-934f-9169e60dc444 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:56.808835 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fae2bf-e372-4a45-934f-9169e60dc444 admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:56.809476 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fae2bf-e372-4a45-934f-9169e60dc444 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:56.811748 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fae2bf-e372-4a45-934f-9169e60dc444 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:56.832365 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-5af58a87-27e3-4167-aa1f-8467eb3b87be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1df1dfa42d674ea8bd1c3c940a2d8b5b'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:56.867088 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1033/2069] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 19:19:56 2022] POST /identity/v3/users => generated 289 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:19:56.870210 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fae2bf-e372-4a45-934f-9169e60dc444 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:56.872849 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-f1fae2bf-e372-4a45-934f-9169e60dc444 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:56.878363 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-5af58a87-27e3-4167-aa1f-8467eb3b87be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1df1dfa42d674ea8bd1c3c940a2d8b5b'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:56.893969 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1037/2070] 10.222.0.61 () {58 vars in 1224 bytes} [Mon Aug 22 19:19:56 2022] DELETE /identity/v3/users/1df1dfa42d674ea8bd1c3c940a2d8b5b => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:19:56.895494 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478726fd-aa3d-4058-a2ba-954f3668b241 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:56.895822 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478726fd-aa3d-4058-a2ba-954f3668b241 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:56.896130 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478726fd-aa3d-4058-a2ba-954f3668b241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:56.900417 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fae2bf-e372-4a45-934f-9169e60dc444 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:56.938677 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-478726fd-aa3d-4058-a2ba-954f3668b241 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:19:56.940003 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-478726fd-aa3d-4058-a2ba-954f3668b241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:57.014001 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:19:57.016511 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a369ce8-de93-4b5d-9b30-65df737313f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:57.017823 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a369ce8-de93-4b5d-9b30-65df737313f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:57.018310 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a369ce8-de93-4b5d-9b30-65df737313f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:57.018657 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a369ce8-de93-4b5d-9b30-65df737313f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:57.019732 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a369ce8-de93-4b5d-9b30-65df737313f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:57.022588 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a369ce8-de93-4b5d-9b30-65df737313f0 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:57.046974 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1034/2071] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:19:56 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:19:57.050482 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a369ce8-de93-4b5d-9b30-65df737313f0 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:57.053954 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1038/2072] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:19:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:19:57.065492 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a369ce8-de93-4b5d-9b30-65df737313f0 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:57.070773 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478726fd-aa3d-4058-a2ba-954f3668b241 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:57.072104 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-478726fd-aa3d-4058-a2ba-954f3668b241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:57.084134 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2002ad98-316a-45a5-8633-51889ab4593f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:57.086796 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2002ad98-316a-45a5-8633-51889ab4593f admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:57.087083 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2002ad98-316a-45a5-8633-51889ab4593f admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:57.087357 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2002ad98-316a-45a5-8633-51889ab4593f admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:57.090256 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2002ad98-316a-45a5-8633-51889ab4593f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:57.093350 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2002ad98-316a-45a5-8633-51889ab4593f admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:57.122798 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1035/2073] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:19:57 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:19:57.123866 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2002ad98-316a-45a5-8633-51889ab4593f admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:57.146959 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2002ad98-316a-45a5-8633-51889ab4593f admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:57.153341 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb4d11e-d6a5-43c9-b067-0773a28b5ed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:57.154025 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb4d11e-d6a5-43c9-b067-0773a28b5ed9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:57.154308 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb4d11e-d6a5-43c9-b067-0773a28b5ed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:57.154661 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb4d11e-d6a5-43c9-b067-0773a28b5ed9 admin admin] PATH_INFO: `/v3/projects/5163a25801f84b9d864413197da52869/users/93daa36b137348d9b3d91e01c2e2c37a/roles/74a40bc819cb42eea7e966a4b2a016f1` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:57.178120 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb4d11e-d6a5-43c9-b067-0773a28b5ed9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5163a25801f84b9d864413197da52869, user_id=93daa36b137348d9b3d91e01c2e2c37a, role_id=74a40bc819cb42eea7e966a4b2a016f1)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:57.181155 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb4d11e-d6a5-43c9-b067-0773a28b5ed9 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:57.185901 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478726fd-aa3d-4058-a2ba-954f3668b241 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:57.190863 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39af0aad-0483-4332-856a-12f452c1d7a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:57.191699 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39af0aad-0483-4332-856a-12f452c1d7a0 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:57.192120 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39af0aad-0483-4332-856a-12f452c1d7a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:57.192631 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39af0aad-0483-4332-856a-12f452c1d7a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:57.203851 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1036/2074] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:19:57 2022] PUT /identity/v3/projects/5163a25801f84b9d864413197da52869/users/93daa36b137348d9b3d91e01c2e2c37a/roles/74a40bc819cb42eea7e966a4b2a016f1 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:19:57.212606 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb4d11e-d6a5-43c9-b067-0773a28b5ed9 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:57.212606 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-ecb4d11e-d6a5-43c9-b067-0773a28b5ed9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:57.217837 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39af0aad-0483-4332-856a-12f452c1d7a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:57.219579 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39af0aad-0483-4332-856a-12f452c1d7a0 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:57.232872 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1039/2075] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:19:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 19:19:57.234221 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0337f3-2725-4045-bc3e-1f2c0cee5d16 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:57.234577 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0337f3-2725-4045-bc3e-1f2c0cee5d16 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:57.234921 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0337f3-2725-4045-bc3e-1f2c0cee5d16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:57.266915 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-4b0337f3-2725-4045-bc3e-1f2c0cee5d16 None None] MFA Rules not processed for user `93daa36b137348d9b3d91e01c2e2c37a`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:19:57.268174 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-4b0337f3-2725-4045-bc3e-1f2c0cee5d16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:57.268923 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:19:57.294924 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb4d11e-d6a5-43c9-b067-0773a28b5ed9 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:57.300416 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d1fa07-03b9-4aff-bc8b-cda534047dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:57.301283 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d1fa07-03b9-4aff-bc8b-cda534047dd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:57.301658 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d1fa07-03b9-4aff-bc8b-cda534047dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:57.302231 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d1fa07-03b9-4aff-bc8b-cda534047dd5 admin admin] PATH_INFO: `/v3/users/844bab0866914f27b0972eb8093b6958` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:57.304638 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d1fa07-03b9-4aff-bc8b-cda534047dd5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=844bab0866914f27b0972eb8093b6958)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:57.307654 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d1fa07-03b9-4aff-bc8b-cda534047dd5 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:57.363306 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1040/2076] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:19:57 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:19:57.367764 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0337f3-2725-4045-bc3e-1f2c0cee5d16 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:57.372307 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-f3d1fa07-03b9-4aff-bc8b-cda534047dd5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '844bab0866914f27b0972eb8093b6958'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:57.373317 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-f3d1fa07-03b9-4aff-bc8b-cda534047dd5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '844bab0866914f27b0972eb8093b6958'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:57.380251 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0337f3-2725-4045-bc3e-1f2c0cee5d16 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:57.382446 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-f3d1fa07-03b9-4aff-bc8b-cda534047dd5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '844bab0866914f27b0972eb8093b6958'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:57.385832 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e53fd9a-bb2c-408e-97f7-70bf0cd34674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:57.386528 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e53fd9a-bb2c-408e-97f7-70bf0cd34674 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:57.386873 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e53fd9a-bb2c-408e-97f7-70bf0cd34674 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:57.387252 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e53fd9a-bb2c-408e-97f7-70bf0cd34674 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:57.388065 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e53fd9a-bb2c-408e-97f7-70bf0cd34674 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:57.390423 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e53fd9a-bb2c-408e-97f7-70bf0cd34674 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:57.393390 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-f3d1fa07-03b9-4aff-bc8b-cda534047dd5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '844bab0866914f27b0972eb8093b6958'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:57.410867 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-f3d1fa07-03b9-4aff-bc8b-cda534047dd5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '844bab0866914f27b0972eb8093b6958'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:57.418392 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1037/2077] 10.222.0.61 () {58 vars in 1224 bytes} [Mon Aug 22 19:19:57 2022] DELETE /identity/v3/users/844bab0866914f27b0972eb8093b6958 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:19:57.422840 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277f98fe-6d75-47c3-9ed9-054472f0541f None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:57.423194 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277f98fe-6d75-47c3-9ed9-054472f0541f None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:57.423548 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277f98fe-6d75-47c3-9ed9-054472f0541f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:57.428648 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1041/2078] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 19:19:57 2022] POST /identity/v3/projects => generated 377 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 19:19:57.428648 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e53fd9a-bb2c-408e-97f7-70bf0cd34674 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:57.432612 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-9e53fd9a-bb2c-408e-97f7-70bf0cd34674 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:57.445686 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-277f98fe-6d75-47c3-9ed9-054472f0541f None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:19:57.447141 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-277f98fe-6d75-47c3-9ed9-054472f0541f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:57.508574 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:19:57.515436 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e53fd9a-bb2c-408e-97f7-70bf0cd34674 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:57.524842 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db0c480-f173-4284-a440-a3a5103b2b64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:57.526591 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db0c480-f173-4284-a440-a3a5103b2b64 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:57.527360 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db0c480-f173-4284-a440-a3a5103b2b64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:57.527818 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db0c480-f173-4284-a440-a3a5103b2b64 admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:57.528862 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db0c480-f173-4284-a440-a3a5103b2b64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:57.531609 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db0c480-f173-4284-a440-a3a5103b2b64 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:57.532227 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1038/2079] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:19:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 19:19:57.542776 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277f98fe-6d75-47c3-9ed9-054472f0541f None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:57.545162 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-277f98fe-6d75-47c3-9ed9-054472f0541f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:57.556271 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1042/2080] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 19:19:57 2022] POST /identity/v3/users => generated 295 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:19:57.560692 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db0c480-f173-4284-a440-a3a5103b2b64 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:57.572920 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db0c480-f173-4284-a440-a3a5103b2b64 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:57.579292 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61abddfa-f5ca-4ff9-b2b4-77a803559e07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:57.580278 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61abddfa-f5ca-4ff9-b2b4-77a803559e07 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:57.580752 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61abddfa-f5ca-4ff9-b2b4-77a803559e07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:57.581080 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61abddfa-f5ca-4ff9-b2b4-77a803559e07 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:57.581741 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61abddfa-f5ca-4ff9-b2b4-77a803559e07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:57.583457 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61abddfa-f5ca-4ff9-b2b4-77a803559e07 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:57.598702 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1043/2081] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:19:57 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:19:57.603300 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61abddfa-f5ca-4ff9-b2b4-77a803559e07 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:57.620506 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61abddfa-f5ca-4ff9-b2b4-77a803559e07 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:57.621740 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277f98fe-6d75-47c3-9ed9-054472f0541f None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:57.626267 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e97498-9116-456a-b5eb-e523265a5344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:57.626718 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349bc41a-4dd1-4aa9-9505-e763a09d8a7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:57.627215 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e97498-9116-456a-b5eb-e523265a5344 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:57.627748 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349bc41a-4dd1-4aa9-9505-e763a09d8a7b service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:57.627890 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e97498-9116-456a-b5eb-e523265a5344 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:57.628317 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349bc41a-4dd1-4aa9-9505-e763a09d8a7b service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:57.628740 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349bc41a-4dd1-4aa9-9505-e763a09d8a7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:57.629890 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e97498-9116-456a-b5eb-e523265a5344 admin admin] PATH_INFO: `/v3/projects/b07bb4a87ad7467cb165c16a4c2f5288/users/981672304b7642b3bd8e9adbc2de29e8/roles/163c30c3b3c246beb849ce176f02d9da` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:57.642675 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349bc41a-4dd1-4aa9-9505-e763a09d8a7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:57.645791 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349bc41a-4dd1-4aa9-9505-e763a09d8a7b service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:57.650333 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e97498-9116-456a-b5eb-e523265a5344 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b07bb4a87ad7467cb165c16a4c2f5288, user_id=981672304b7642b3bd8e9adbc2de29e8, role_id=163c30c3b3c246beb849ce176f02d9da)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:57.657098 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e97498-9116-456a-b5eb-e523265a5344 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:57.668990 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1039/2082] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:19:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 19:19:57.682670 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1044/2083] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:19:57 2022] PUT /identity/v3/projects/b07bb4a87ad7467cb165c16a4c2f5288/users/981672304b7642b3bd8e9adbc2de29e8/roles/163c30c3b3c246beb849ce176f02d9da => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:19:57.687252 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349bc41a-4dd1-4aa9-9505-e763a09d8a7b service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:57.693067 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:19:57.705759 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349bc41a-4dd1-4aa9-9505-e763a09d8a7b service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:57.814228 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1515485-f1ef-4b2b-b7a1-3d8c906e182f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:57.814849 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1515485-f1ef-4b2b-b7a1-3d8c906e182f admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:57.815299 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1515485-f1ef-4b2b-b7a1-3d8c906e182f admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:57.816135 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1515485-f1ef-4b2b-b7a1-3d8c906e182f admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:57.816804 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1515485-f1ef-4b2b-b7a1-3d8c906e182f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:57.818438 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1515485-f1ef-4b2b-b7a1-3d8c906e182f admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:57.831677 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1040/2084] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:19:57 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:19:57.834155 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e97498-9116-456a-b5eb-e523265a5344 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:57.845786 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e97498-9116-456a-b5eb-e523265a5344 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:57.851126 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60074413-113c-44ae-a14b-02c420a93392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:57.852160 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60074413-113c-44ae-a14b-02c420a93392 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:57.852523 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60074413-113c-44ae-a14b-02c420a93392 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:57.852702 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60074413-113c-44ae-a14b-02c420a93392 admin admin] PATH_INFO: `/v3/projects/b07bb4a87ad7467cb165c16a4c2f5288/users/981672304b7642b3bd8e9adbc2de29e8/roles/163c30c3b3c246beb849ce176f02d9da` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:57.854944 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60074413-113c-44ae-a14b-02c420a93392 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b07bb4a87ad7467cb165c16a4c2f5288, user_id=981672304b7642b3bd8e9adbc2de29e8, role_id=163c30c3b3c246beb849ce176f02d9da)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:57.857615 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60074413-113c-44ae-a14b-02c420a93392 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:57.869453 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1045/2085] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:19:57 2022] PUT /identity/v3/projects/b07bb4a87ad7467cb165c16a4c2f5288/users/981672304b7642b3bd8e9adbc2de29e8/roles/163c30c3b3c246beb849ce176f02d9da => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:19:57.879586 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47234a88-e27b-46c5-af6d-ff18d2d307bb None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:57.879901 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47234a88-e27b-46c5-af6d-ff18d2d307bb None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:57.880177 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47234a88-e27b-46c5-af6d-ff18d2d307bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:57.914996 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-47234a88-e27b-46c5-af6d-ff18d2d307bb None None] MFA Rules not processed for user `981672304b7642b3bd8e9adbc2de29e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:19:57.916417 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-47234a88-e27b-46c5-af6d-ff18d2d307bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:57.991338 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60074413-113c-44ae-a14b-02c420a93392 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:57.994175 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-60074413-113c-44ae-a14b-02c420a93392 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:58.041765 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1041/2086] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:19:57 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:19:58.049539 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47234a88-e27b-46c5-af6d-ff18d2d307bb None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:58.051273 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-47234a88-e27b-46c5-af6d-ff18d2d307bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:58.120583 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60074413-113c-44ae-a14b-02c420a93392 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:58.127692 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47234a88-e27b-46c5-af6d-ff18d2d307bb None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:58.127922 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac86660d-57df-4d64-8e3d-72631eb9ddf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:58.128907 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac86660d-57df-4d64-8e3d-72631eb9ddf0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:58.129183 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac86660d-57df-4d64-8e3d-72631eb9ddf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:58.130448 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac86660d-57df-4d64-8e3d-72631eb9ddf0 admin admin] PATH_INFO: `/v3/projects/cc9a19ea3d62466493d02a9f96cb154a` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:58.131817 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac86660d-57df-4d64-8e3d-72631eb9ddf0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc9a19ea3d62466493d02a9f96cb154a)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:58.132288 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd4bf7b-322b-4f27-a113-751f60b18286 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e79498873d4ed29a0e19bbea9b9214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:58.132905 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd4bf7b-322b-4f27-a113-751f60b18286 service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:58.133755 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac86660d-57df-4d64-8e3d-72631eb9ddf0 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:58.134291 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd4bf7b-322b-4f27-a113-751f60b18286 service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:58.134574 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd4bf7b-322b-4f27-a113-751f60b18286 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:58.136919 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-9cd4bf7b-322b-4f27-a113-751f60b18286 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:58.193842 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:19:58.211087 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-ac86660d-57df-4d64-8e3d-72631eb9ddf0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc9a19ea3d62466493d02a9f96cb154a', 'request_id': 'req-ac86660d-57df-4d64-8e3d-72631eb9ddf0'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:58.233291 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-ac86660d-57df-4d64-8e3d-72631eb9ddf0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc9a19ea3d62466493d02a9f96cb154a', 'request_id': 'req-ac86660d-57df-4d64-8e3d-72631eb9ddf0'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:58.239274 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd4bf7b-322b-4f27-a113-751f60b18286 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:58.241049 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-ac86660d-57df-4d64-8e3d-72631eb9ddf0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc9a19ea3d62466493d02a9f96cb154a', 'request_id': 'req-ac86660d-57df-4d64-8e3d-72631eb9ddf0'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:58.244194 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd4bf7b-322b-4f27-a113-751f60b18286 service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:58.259535 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-ac86660d-57df-4d64-8e3d-72631eb9ddf0 admin admin] The token cache is being invalidate because project cc9a19ea3d62466493d02a9f96cb154a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 19:19:58.259997 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-ac86660d-57df-4d64-8e3d-72631eb9ddf0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:58.260937 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-ac86660d-57df-4d64-8e3d-72631eb9ddf0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:58.262545 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1046/2087] 10.222.0.61 () {58 vars in 1233 bytes} [Mon Aug 22 19:19:57 2022] DELETE /identity/v3/projects/cc9a19ea3d62466493d02a9f96cb154a => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:19:58.266755 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac86660d-57df-4d64-8e3d-72631eb9ddf0 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:58.267209 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1042/2088] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:19:58 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 19:19:58.269468 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-ac86660d-57df-4d64-8e3d-72631eb9ddf0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:58.317228 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:19:58.374531 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac86660d-57df-4d64-8e3d-72631eb9ddf0 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:58.382702 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c32241-6a95-41dc-b3b9-27efdfa54276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:58.384085 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c32241-6a95-41dc-b3b9-27efdfa54276 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:58.384515 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c32241-6a95-41dc-b3b9-27efdfa54276 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:58.385278 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c32241-6a95-41dc-b3b9-27efdfa54276 admin admin] PATH_INFO: `/v3/users/bec2fba8388f4a039d2d663c01aa04e4` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:58.386842 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c32241-6a95-41dc-b3b9-27efdfa54276 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bec2fba8388f4a039d2d663c01aa04e4)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:58.389969 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c32241-6a95-41dc-b3b9-27efdfa54276 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:58.506769 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-a2c32241-6a95-41dc-b3b9-27efdfa54276 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bec2fba8388f4a039d2d663c01aa04e4'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:58.507597 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-a2c32241-6a95-41dc-b3b9-27efdfa54276 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bec2fba8388f4a039d2d663c01aa04e4'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:58.516444 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-a2c32241-6a95-41dc-b3b9-27efdfa54276 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bec2fba8388f4a039d2d663c01aa04e4'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:58.532025 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-a2c32241-6a95-41dc-b3b9-27efdfa54276 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bec2fba8388f4a039d2d663c01aa04e4'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:58.563610 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-a2c32241-6a95-41dc-b3b9-27efdfa54276 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bec2fba8388f4a039d2d663c01aa04e4'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:58.576775 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1047/2089] 10.222.0.61 () {58 vars in 1224 bytes} [Mon Aug 22 19:19:58 2022] DELETE /identity/v3/users/bec2fba8388f4a039d2d663c01aa04e4 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:19:58.756924 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd4bf7b-322b-4f27-a113-751f60b18286 service cinder] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:58.758795 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-9cd4bf7b-322b-4f27-a113-751f60b18286 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:58.778718 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd4bf7b-322b-4f27-a113-751f60b18286 service cinder] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:58.870351 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8266ddcc-475b-49ed-8534-4999eb940c6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:58.870926 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8266ddcc-475b-49ed-8534-4999eb940c6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:58.871210 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8266ddcc-475b-49ed-8534-4999eb940c6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:58.871481 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8266ddcc-475b-49ed-8534-4999eb940c6e admin admin] PATH_INFO: `/v3/projects/8ce7437988394d0aa37f4a7bb3ecba39` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:58.872814 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8266ddcc-475b-49ed-8534-4999eb940c6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ce7437988394d0aa37f4a7bb3ecba39)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:58.875372 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8266ddcc-475b-49ed-8534-4999eb940c6e admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:58.935755 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-8266ddcc-475b-49ed-8534-4999eb940c6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ce7437988394d0aa37f4a7bb3ecba39', 'request_id': 'req-8266ddcc-475b-49ed-8534-4999eb940c6e'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:58.944891 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-8266ddcc-475b-49ed-8534-4999eb940c6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ce7437988394d0aa37f4a7bb3ecba39', 'request_id': 'req-8266ddcc-475b-49ed-8534-4999eb940c6e'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:58.952511 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-8266ddcc-475b-49ed-8534-4999eb940c6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ce7437988394d0aa37f4a7bb3ecba39', 'request_id': 'req-8266ddcc-475b-49ed-8534-4999eb940c6e'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:58.967409 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-8266ddcc-475b-49ed-8534-4999eb940c6e admin admin] The token cache is being invalidate because project 8ce7437988394d0aa37f4a7bb3ecba39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 19:19:58.967727 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-8266ddcc-475b-49ed-8534-4999eb940c6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:58.968391 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-8266ddcc-475b-49ed-8534-4999eb940c6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:58.970233 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1043/2090] 10.222.0.61 () {58 vars in 1233 bytes} [Mon Aug 22 19:19:58 2022] DELETE /identity/v3/projects/8ce7437988394d0aa37f4a7bb3ecba39 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:19:58.974771 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3eb485-ba54-4707-b724-18ad840075fc None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:58.975076 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3eb485-ba54-4707-b724-18ad840075fc None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:58.975338 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3eb485-ba54-4707-b724-18ad840075fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:59.007954 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-4f3eb485-ba54-4707-b724-18ad840075fc None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:19:59.010623 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-4f3eb485-ba54-4707-b724-18ad840075fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:59.047629 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8266ddcc-475b-49ed-8534-4999eb940c6e admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:59.049723 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-8266ddcc-475b-49ed-8534-4999eb940c6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:59.111829 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1048/2091] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:19:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:19:59.114479 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1a3204-f115-4e73-a5e8-357d4752a90e None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:59.114871 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1a3204-f115-4e73-a5e8-357d4752a90e None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:59.115251 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1a3204-f115-4e73-a5e8-357d4752a90e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:59.138760 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-4b1a3204-f115-4e73-a5e8-357d4752a90e None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:19:59.140183 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-4b1a3204-f115-4e73-a5e8-357d4752a90e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:59.141835 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8266ddcc-475b-49ed-8534-4999eb940c6e admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:59.151740 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1049/2092] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:19:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:19:59.151910 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6064e39-80fb-4b7c-8bdc-7f078288f0ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:59.151910 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6064e39-80fb-4b7c-8bdc-7f078288f0ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:59.151910 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6064e39-80fb-4b7c-8bdc-7f078288f0ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:59.151910 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6064e39-80fb-4b7c-8bdc-7f078288f0ef admin admin] PATH_INFO: `/v3/projects/9cf65c3fd0b643d990edff75349bcae6` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:59.158456 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6064e39-80fb-4b7c-8bdc-7f078288f0ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=9cf65c3fd0b643d990edff75349bcae6)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:59.159867 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1a3204-f115-4e73-a5e8-357d4752a90e None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:59.161137 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6064e39-80fb-4b7c-8bdc-7f078288f0ef admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:59.172264 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1a3204-f115-4e73-a5e8-357d4752a90e None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:59.175564 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:19:59.182661 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43077248-5637-404a-9a02-f86e9e009f81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:59.184961 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43077248-5637-404a-9a02-f86e9e009f81 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:59.185468 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43077248-5637-404a-9a02-f86e9e009f81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:59.186101 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43077248-5637-404a-9a02-f86e9e009f81 admin admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:59.187007 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43077248-5637-404a-9a02-f86e9e009f81 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:19:59.188052 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43077248-5637-404a-9a02-f86e9e009f81 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:59.191563 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43077248-5637-404a-9a02-f86e9e009f81 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:59.224150 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1050/2093] 10.222.0.61 () {58 vars in 1153 bytes} [Mon Aug 22 19:19:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:19:59.228521 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43077248-5637-404a-9a02-f86e9e009f81 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:59.249270 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43077248-5637-404a-9a02-f86e9e009f81 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:59.250332 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-c6064e39-80fb-4b7c-8bdc-7f078288f0ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9cf65c3fd0b643d990edff75349bcae6', 'request_id': 'req-c6064e39-80fb-4b7c-8bdc-7f078288f0ef'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:59.262389 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-c6064e39-80fb-4b7c-8bdc-7f078288f0ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9cf65c3fd0b643d990edff75349bcae6', 'request_id': 'req-c6064e39-80fb-4b7c-8bdc-7f078288f0ef'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:59.273546 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-c6064e39-80fb-4b7c-8bdc-7f078288f0ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9cf65c3fd0b643d990edff75349bcae6', 'request_id': 'req-c6064e39-80fb-4b7c-8bdc-7f078288f0ef'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:59.292620 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-c6064e39-80fb-4b7c-8bdc-7f078288f0ef admin admin] The token cache is being invalidate because project 9cf65c3fd0b643d990edff75349bcae6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 19:19:59.293096 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-c6064e39-80fb-4b7c-8bdc-7f078288f0ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:59.293883 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:19:59.294535 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-c6064e39-80fb-4b7c-8bdc-7f078288f0ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:19:59.298072 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1044/2094] 10.222.0.61 () {58 vars in 1233 bytes} [Mon Aug 22 19:19:59 2022] DELETE /identity/v3/projects/9cf65c3fd0b643d990edff75349bcae6 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:19:59.303998 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4ed7ad-4ede-4bc5-ae76-20d82ca92ff0 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:59.304285 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4ed7ad-4ede-4bc5-ae76-20d82ca92ff0 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:59.304554 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4ed7ad-4ede-4bc5-ae76-20d82ca92ff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:59.345616 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9883c279-77ad-4da0-8143-ac583b5ade35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:59.346162 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9883c279-77ad-4da0-8143-ac583b5ade35 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:59.346356 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-ae4ed7ad-4ede-4bc5-ae76-20d82ca92ff0 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:19:59.346486 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9883c279-77ad-4da0-8143-ac583b5ade35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:59.346657 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9883c279-77ad-4da0-8143-ac583b5ade35 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:59.347284 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9883c279-77ad-4da0-8143-ac583b5ade35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:59.348727 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-ae4ed7ad-4ede-4bc5-ae76-20d82ca92ff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:59.349791 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9883c279-77ad-4da0-8143-ac583b5ade35 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:59.371849 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1045/2095] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:19:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:19:59.373370 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc6efc9-4852-4e21-94c7-a5c3082a39ef None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:59.373657 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc6efc9-4852-4e21-94c7-a5c3082a39ef None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:59.373956 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc6efc9-4852-4e21-94c7-a5c3082a39ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:59.382571 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1051/2096] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 19:19:59 2022] POST /identity/v3/projects => generated 375 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:19:59.387316 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9883c279-77ad-4da0-8143-ac583b5ade35 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:59.389047 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-9883c279-77ad-4da0-8143-ac583b5ade35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:59.401678 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-ecc6efc9-4852-4e21-94c7-a5c3082a39ef None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:19:59.403038 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-ecc6efc9-4852-4e21-94c7-a5c3082a39ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:19:59.484769 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1046/2097] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:19:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:19:59.490130 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc6efc9-4852-4e21-94c7-a5c3082a39ef None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:59.501298 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9883c279-77ad-4da0-8143-ac583b5ade35 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:59.513365 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18adfcaf-a274-4ba4-af46-485ccfc85c53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:59.513997 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18adfcaf-a274-4ba4-af46-485ccfc85c53 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:59.514271 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18adfcaf-a274-4ba4-af46-485ccfc85c53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:59.514543 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18adfcaf-a274-4ba4-af46-485ccfc85c53 admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:59.515176 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18adfcaf-a274-4ba4-af46-485ccfc85c53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:59.518489 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc6efc9-4852-4e21-94c7-a5c3082a39ef None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:59.522492 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18adfcaf-a274-4ba4-af46-485ccfc85c53 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:59.529384 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4e1a30-c73b-445d-a33d-2d74cf39a80b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:59.529729 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4e1a30-c73b-445d-a33d-2d74cf39a80b admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:59.530009 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4e1a30-c73b-445d-a33d-2d74cf39a80b admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:59.530282 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4e1a30-c73b-445d-a33d-2d74cf39a80b admin admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:59.530955 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4e1a30-c73b-445d-a33d-2d74cf39a80b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:19:59.531257 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4e1a30-c73b-445d-a33d-2d74cf39a80b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:59.534013 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4e1a30-c73b-445d-a33d-2d74cf39a80b admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:59.548210 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1047/2098] 10.222.0.61 () {58 vars in 1153 bytes} [Mon Aug 22 19:19:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:19:59.552723 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4e1a30-c73b-445d-a33d-2d74cf39a80b admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:59.567306 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1052/2099] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 19:19:59 2022] POST /identity/v3/users => generated 288 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 19:19:59.571313 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18adfcaf-a274-4ba4-af46-485ccfc85c53 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:59.573730 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4e1a30-c73b-445d-a33d-2d74cf39a80b admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:59.579558 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3dc631-0817-4e07-bf62-f7ac004b2182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:59.581316 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3dc631-0817-4e07-bf62-f7ac004b2182 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:59.581655 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3dc631-0817-4e07-bf62-f7ac004b2182 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:59.581935 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3dc631-0817-4e07-bf62-f7ac004b2182 admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:59.582767 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3dc631-0817-4e07-bf62-f7ac004b2182 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:59.584937 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18adfcaf-a274-4ba4-af46-485ccfc85c53 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:59.585933 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3dc631-0817-4e07-bf62-f7ac004b2182 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:59.591967 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8297d76b-d571-4c96-aecf-a87ac7673cf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:59.592781 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8297d76b-d571-4c96-aecf-a87ac7673cf6 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:59.593128 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8297d76b-d571-4c96-aecf-a87ac7673cf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:59.593470 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8297d76b-d571-4c96-aecf-a87ac7673cf6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:59.594193 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8297d76b-d571-4c96-aecf-a87ac7673cf6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:59.596629 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8297d76b-d571-4c96-aecf-a87ac7673cf6 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:59.610496 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1053/2100] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:19:59 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 19:19:59.625747 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8297d76b-d571-4c96-aecf-a87ac7673cf6 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:59.645905 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1048/2101] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 19:19:59 2022] POST /identity/v3/projects => generated 377 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 19:19:59.649716 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3dc631-0817-4e07-bf62-f7ac004b2182 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:59.660840 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8297d76b-d571-4c96-aecf-a87ac7673cf6 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:59.784880 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:19:59.785619 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b25ad6-a933-4360-b226-e7b0240c3662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:59.786159 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b25ad6-a933-4360-b226-e7b0240c3662 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:59.786482 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b25ad6-a933-4360-b226-e7b0240c3662 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:59.786760 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b25ad6-a933-4360-b226-e7b0240c3662 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:59.788217 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b25ad6-a933-4360-b226-e7b0240c3662 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:59.790012 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b25ad6-a933-4360-b226-e7b0240c3662 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:59.794738 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3dc631-0817-4e07-bf62-f7ac004b2182 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:59.803371 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61d3c72-8c7d-4f54-b3f3-2f824276a580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:59.805924 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61d3c72-8c7d-4f54-b3f3-2f824276a580 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:59.806627 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61d3c72-8c7d-4f54-b3f3-2f824276a580 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:59.807656 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61d3c72-8c7d-4f54-b3f3-2f824276a580 admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:59.808603 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61d3c72-8c7d-4f54-b3f3-2f824276a580 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:59.809801 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1054/2102] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:19:59 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 19:19:59.810865 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b25ad6-a933-4360-b226-e7b0240c3662 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:59.815626 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61d3c72-8c7d-4f54-b3f3-2f824276a580 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:59.838647 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b25ad6-a933-4360-b226-e7b0240c3662 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:59.848654 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a56002-0efe-4117-8d17-815f35543983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:59.850082 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1049/2103] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 19:19:59 2022] POST /identity/v3/users => generated 289 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:19:59.850625 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a56002-0efe-4117-8d17-815f35543983 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:59.850916 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a56002-0efe-4117-8d17-815f35543983 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:59.851195 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a56002-0efe-4117-8d17-815f35543983 admin admin] PATH_INFO: `/v3/projects/efed7e85274942b0b62d1c649c9a7619/users/6e22c47f99f44cb5b63a5f55ef712813/roles/74a40bc819cb42eea7e966a4b2a016f1` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:59.855717 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61d3c72-8c7d-4f54-b3f3-2f824276a580 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:59.870943 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61d3c72-8c7d-4f54-b3f3-2f824276a580 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:59.879202 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e50d23a-e562-433c-8a39-48cae21054e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:19:59.880667 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e50d23a-e562-433c-8a39-48cae21054e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:59.881183 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e50d23a-e562-433c-8a39-48cae21054e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:59.881554 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e50d23a-e562-433c-8a39-48cae21054e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:59.883347 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e50d23a-e562-433c-8a39-48cae21054e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:59.885427 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e50d23a-e562-433c-8a39-48cae21054e7 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:59.890945 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a56002-0efe-4117-8d17-815f35543983 admin admin] RBAC: Authorizing `identity:create_grant(project_id=efed7e85274942b0b62d1c649c9a7619, user_id=6e22c47f99f44cb5b63a5f55ef712813, role_id=74a40bc819cb42eea7e966a4b2a016f1)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:19:59.899807 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1050/2104] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:19:59 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:19:59.904994 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e50d23a-e562-433c-8a39-48cae21054e7 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:19:59.909084 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a56002-0efe-4117-8d17-815f35543983 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:19:59.923914 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e50d23a-e562-433c-8a39-48cae21054e7 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:19:59.937084 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1055/2105] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:19:59 2022] PUT /identity/v3/projects/efed7e85274942b0b62d1c649c9a7619/users/6e22c47f99f44cb5b63a5f55ef712813/roles/74a40bc819cb42eea7e966a4b2a016f1 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:19:59.942166 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61517e8f-b54a-40eb-88f6-41c462fb9d54 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:19:59.942166 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61517e8f-b54a-40eb-88f6-41c462fb9d54 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:19:59.942469 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61517e8f-b54a-40eb-88f6-41c462fb9d54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:19:59.950069 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:19:59.981004 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-61517e8f-b54a-40eb-88f6-41c462fb9d54 None None] MFA Rules not processed for user `6e22c47f99f44cb5b63a5f55ef712813`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:19:59.982274 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-61517e8f-b54a-40eb-88f6-41c462fb9d54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:20:00.027560 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726e4b02-f42f-405d-b046-e682f33d4fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:00.028251 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726e4b02-f42f-405d-b046-e682f33d4fcc admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:00.028529 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726e4b02-f42f-405d-b046-e682f33d4fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:00.028808 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726e4b02-f42f-405d-b046-e682f33d4fcc admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:00.029654 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726e4b02-f42f-405d-b046-e682f33d4fcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:00.032136 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726e4b02-f42f-405d-b046-e682f33d4fcc admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:00.050017 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1051/2106] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:19:59 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:20:00.051487 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726e4b02-f42f-405d-b046-e682f33d4fcc admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:00.065809 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726e4b02-f42f-405d-b046-e682f33d4fcc admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:00.069567 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1056/2107] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:19:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 19:20:00.071043 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61517e8f-b54a-40eb-88f6-41c462fb9d54 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:00.073275 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9c92da-46d7-4a63-9419-42536c388682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:00.074063 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9c92da-46d7-4a63-9419-42536c388682 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:00.074447 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9c92da-46d7-4a63-9419-42536c388682 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:00.074841 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9c92da-46d7-4a63-9419-42536c388682 admin admin] PATH_INFO: `/v3/projects/1815855da8ee442bb4674d155de57b7f/users/8682aca1fc4f4b8a9e244e84b3676d94/roles/74a40bc819cb42eea7e966a4b2a016f1` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:00.087312 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61517e8f-b54a-40eb-88f6-41c462fb9d54 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:00.094030 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c8184f-bffe-4beb-88ac-7819e2d7f58b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:00.094879 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c8184f-bffe-4beb-88ac-7819e2d7f58b admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:00.095166 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c8184f-bffe-4beb-88ac-7819e2d7f58b admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:00.095451 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c8184f-bffe-4beb-88ac-7819e2d7f58b admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:00.095795 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9c92da-46d7-4a63-9419-42536c388682 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1815855da8ee442bb4674d155de57b7f, user_id=8682aca1fc4f4b8a9e244e84b3676d94, role_id=74a40bc819cb42eea7e966a4b2a016f1)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:00.096124 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c8184f-bffe-4beb-88ac-7819e2d7f58b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:00.098020 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c8184f-bffe-4beb-88ac-7819e2d7f58b admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:00.099033 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9c92da-46d7-4a63-9419-42536c388682 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:00.124004 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1052/2108] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:20:00 2022] PUT /identity/v3/projects/1815855da8ee442bb4674d155de57b7f/users/8682aca1fc4f4b8a9e244e84b3676d94/roles/74a40bc819cb42eea7e966a4b2a016f1 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:20:00.133871 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af179be-29d0-492a-a94c-a9880cc3e29a None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:00.134159 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af179be-29d0-492a-a94c-a9880cc3e29a None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:00.134433 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af179be-29d0-492a-a94c-a9880cc3e29a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:00.153764 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1057/2109] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 19:20:00 2022] POST /identity/v3/projects => generated 375 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:20:00.155107 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c8184f-bffe-4beb-88ac-7819e2d7f58b admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:00.181538 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-8af179be-29d0-492a-a94c-a9880cc3e29a None None] MFA Rules not processed for user `8682aca1fc4f4b8a9e244e84b3676d94`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:20:00.183573 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-8af179be-29d0-492a-a94c-a9880cc3e29a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:20:00.232298 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c8184f-bffe-4beb-88ac-7819e2d7f58b admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:00.237659 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f3d9ae-5798-4f83-bd4a-3dedaad8dac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:00.238288 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f3d9ae-5798-4f83-bd4a-3dedaad8dac4 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:00.238570 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f3d9ae-5798-4f83-bd4a-3dedaad8dac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:00.238840 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f3d9ae-5798-4f83-bd4a-3dedaad8dac4 admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:00.239501 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f3d9ae-5798-4f83-bd4a-3dedaad8dac4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:00.241485 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f3d9ae-5798-4f83-bd4a-3dedaad8dac4 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:00.269685 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1058/2110] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 19:20:00 2022] POST /identity/v3/users => generated 294 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:20:00.273795 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f3d9ae-5798-4f83-bd4a-3dedaad8dac4 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:00.284276 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f3d9ae-5798-4f83-bd4a-3dedaad8dac4 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:00.289983 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fe0a44-7032-4303-9019-f7f0a05e3a97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:00.290602 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fe0a44-7032-4303-9019-f7f0a05e3a97 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:00.290880 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fe0a44-7032-4303-9019-f7f0a05e3a97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:00.291733 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fe0a44-7032-4303-9019-f7f0a05e3a97 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:00.292394 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fe0a44-7032-4303-9019-f7f0a05e3a97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:00.294018 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fe0a44-7032-4303-9019-f7f0a05e3a97 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:00.294703 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1053/2111] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:20:00 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:20:00.305374 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1059/2112] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:20:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 19:20:00.309986 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af179be-29d0-492a-a94c-a9880cc3e29a None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:00.311125 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fe0a44-7032-4303-9019-f7f0a05e3a97 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:00.312887 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-30fe0a44-7032-4303-9019-f7f0a05e3a97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:20:00.324298 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af179be-29d0-492a-a94c-a9880cc3e29a None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:00.330457 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98be527b-9f2b-4c66-8041-5ea4d9b4994d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:00.331265 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98be527b-9f2b-4c66-8041-5ea4d9b4994d admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:00.331661 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98be527b-9f2b-4c66-8041-5ea4d9b4994d admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:00.332016 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98be527b-9f2b-4c66-8041-5ea4d9b4994d admin admin] PATH_INFO: `/v3/projects/a78febef45b144148fb43c42bcca5fd0/users/e6ffd9b417d74c1cbc51a47ef99679f2/roles/163c30c3b3c246beb849ce176f02d9da` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:00.352679 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98be527b-9f2b-4c66-8041-5ea4d9b4994d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a78febef45b144148fb43c42bcca5fd0, user_id=e6ffd9b417d74c1cbc51a47ef99679f2, role_id=163c30c3b3c246beb849ce176f02d9da)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:00.356071 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98be527b-9f2b-4c66-8041-5ea4d9b4994d admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:00.374278 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1054/2113] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:20:00 2022] PUT /identity/v3/projects/a78febef45b144148fb43c42bcca5fd0/users/e6ffd9b417d74c1cbc51a47ef99679f2/roles/163c30c3b3c246beb849ce176f02d9da => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:20:00.379835 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98be527b-9f2b-4c66-8041-5ea4d9b4994d admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:00.398206 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:20:00.402949 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:20:00.456735 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fe0a44-7032-4303-9019-f7f0a05e3a97 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:00.483290 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98be527b-9f2b-4c66-8041-5ea4d9b4994d admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:00.488777 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895d4372-74b8-4539-88bb-7ceeb5b83cf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:00.489489 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895d4372-74b8-4539-88bb-7ceeb5b83cf3 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:00.489835 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895d4372-74b8-4539-88bb-7ceeb5b83cf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:00.490106 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895d4372-74b8-4539-88bb-7ceeb5b83cf3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:00.490763 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895d4372-74b8-4539-88bb-7ceeb5b83cf3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:00.492474 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895d4372-74b8-4539-88bb-7ceeb5b83cf3 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:00.509595 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1055/2114] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:20:00 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:20:00.511250 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895d4372-74b8-4539-88bb-7ceeb5b83cf3 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:00.522655 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895d4372-74b8-4539-88bb-7ceeb5b83cf3 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:00.527760 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8789aca8-4baa-454f-8eae-4c461165308f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:00.528432 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8789aca8-4baa-454f-8eae-4c461165308f admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:00.528706 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8789aca8-4baa-454f-8eae-4c461165308f admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:00.528980 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8789aca8-4baa-454f-8eae-4c461165308f admin admin] PATH_INFO: `/v3/projects/a78febef45b144148fb43c42bcca5fd0/users/e6ffd9b417d74c1cbc51a47ef99679f2/roles/163c30c3b3c246beb849ce176f02d9da` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:00.531534 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8789aca8-4baa-454f-8eae-4c461165308f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a78febef45b144148fb43c42bcca5fd0, user_id=e6ffd9b417d74c1cbc51a47ef99679f2, role_id=163c30c3b3c246beb849ce176f02d9da)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:00.534369 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8789aca8-4baa-454f-8eae-4c461165308f admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:00.535560 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ff7590-97b5-40c3-ad68-2c332a3653ad service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d6a79ed44e5396d13cd29ba160ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:00.536408 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ff7590-97b5-40c3-ad68-2c332a3653ad service glance] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:00.536895 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ff7590-97b5-40c3-ad68-2c332a3653ad service glance] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:00.537518 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ff7590-97b5-40c3-ad68-2c332a3653ad service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:00.549763 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1056/2115] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:20:00 2022] PUT /identity/v3/projects/a78febef45b144148fb43c42bcca5fd0/users/e6ffd9b417d74c1cbc51a47ef99679f2/roles/163c30c3b3c246beb849ce176f02d9da => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:20:00.561381 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bd30ad-208f-455a-a821-bb7648853b99 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:00.562517 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bd30ad-208f-455a-a821-bb7648853b99 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:00.562801 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bd30ad-208f-455a-a821-bb7648853b99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:00.596990 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-68bd30ad-208f-455a-a821-bb7648853b99 None None] MFA Rules not processed for user `e6ffd9b417d74c1cbc51a47ef99679f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:20:00.598460 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-68bd30ad-208f-455a-a821-bb7648853b99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:20:00.619630 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ff7590-97b5-40c3-ad68-2c332a3653ad service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:00.622183 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ff7590-97b5-40c3-ad68-2c332a3653ad service glance] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:00.635420 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1060/2116] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:20:00 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 19:20:00.751738 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1057/2117] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:20:00 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:20:00.761371 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ff7590-97b5-40c3-ad68-2c332a3653ad service glance] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:00.762996 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-50ff7590-97b5-40c3-ad68-2c332a3653ad service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:20:00.878798 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ff7590-97b5-40c3-ad68-2c332a3653ad service glance] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:00.968844 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74479e04-5952-4c6c-b36c-80b9e7cff4d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e79498873d4ed29a0e19bbea9b9214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:00.970068 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74479e04-5952-4c6c-b36c-80b9e7cff4d4 service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:00.970349 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74479e04-5952-4c6c-b36c-80b9e7cff4d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:00.970619 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74479e04-5952-4c6c-b36c-80b9e7cff4d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:00.981488 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:20:01.053035 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74479e04-5952-4c6c-b36c-80b9e7cff4d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:01.055657 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74479e04-5952-4c6c-b36c-80b9e7cff4d4 service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:01.074537 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1061/2118] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:20:00 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:20:01.683726 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bd30ad-208f-455a-a821-bb7648853b99 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:01.685402 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-68bd30ad-208f-455a-a821-bb7648853b99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:20:01.719392 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bd30ad-208f-455a-a821-bb7648853b99 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:01.734837 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daf7cdb-b7ef-4da5-856b-76a09089e4aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e79498873d4ed29a0e19bbea9b9214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:01.735627 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daf7cdb-b7ef-4da5-856b-76a09089e4aa service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:01.735889 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daf7cdb-b7ef-4da5-856b-76a09089e4aa service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:01.736339 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daf7cdb-b7ef-4da5-856b-76a09089e4aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:01.740950 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-0daf7cdb-b7ef-4da5-856b-76a09089e4aa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:20:01.833363 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daf7cdb-b7ef-4da5-856b-76a09089e4aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:01.835947 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daf7cdb-b7ef-4da5-856b-76a09089e4aa service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:01.851226 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1058/2119] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:20:01 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:20:02.671904 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74479e04-5952-4c6c-b36c-80b9e7cff4d4 service cinder] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:02.681051 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74479e04-5952-4c6c-b36c-80b9e7cff4d4 service cinder] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:02.686328 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76ba5eb-1a8a-4cf4-ad6e-7b37062b3ad2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e79498873d4ed29a0e19bbea9b9214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:02.687691 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76ba5eb-1a8a-4cf4-ad6e-7b37062b3ad2 service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:02.688369 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76ba5eb-1a8a-4cf4-ad6e-7b37062b3ad2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:02.689203 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76ba5eb-1a8a-4cf4-ad6e-7b37062b3ad2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:02.691330 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-b76ba5eb-1a8a-4cf4-ad6e-7b37062b3ad2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:20:02.783178 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76ba5eb-1a8a-4cf4-ad6e-7b37062b3ad2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:02.784998 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76ba5eb-1a8a-4cf4-ad6e-7b37062b3ad2 service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:02.797092 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1062/2120] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:20:02 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:20:04.184009 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daf7cdb-b7ef-4da5-856b-76a09089e4aa service cinder] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:04.206022 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:20:04.297096 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daf7cdb-b7ef-4da5-856b-76a09089e4aa service cinder] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:04.301842 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64168114-f9ff-4829-8118-6c503c829755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:04.302478 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64168114-f9ff-4829-8118-6c503c829755 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:04.302779 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64168114-f9ff-4829-8118-6c503c829755 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:04.303048 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64168114-f9ff-4829-8118-6c503c829755 admin admin] PATH_INFO: `/v3/users/6e22c47f99f44cb5b63a5f55ef712813` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:04.304213 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64168114-f9ff-4829-8118-6c503c829755 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e22c47f99f44cb5b63a5f55ef712813)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:04.306113 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64168114-f9ff-4829-8118-6c503c829755 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:04.355813 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-64168114-f9ff-4829-8118-6c503c829755 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e22c47f99f44cb5b63a5f55ef712813'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:04.356432 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-64168114-f9ff-4829-8118-6c503c829755 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e22c47f99f44cb5b63a5f55ef712813'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:04.368630 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-64168114-f9ff-4829-8118-6c503c829755 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e22c47f99f44cb5b63a5f55ef712813'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:04.381806 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-64168114-f9ff-4829-8118-6c503c829755 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e22c47f99f44cb5b63a5f55ef712813'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:04.395681 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-64168114-f9ff-4829-8118-6c503c829755 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e22c47f99f44cb5b63a5f55ef712813'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:04.402165 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1059/2121] 10.222.0.61 () {58 vars in 1224 bytes} [Mon Aug 22 19:20:04 2022] DELETE /identity/v3/users/6e22c47f99f44cb5b63a5f55ef712813 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:20:04.406610 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525bb371-3fc5-4b96-af3e-553b5903d843 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:04.407154 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525bb371-3fc5-4b96-af3e-553b5903d843 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:04.407266 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525bb371-3fc5-4b96-af3e-553b5903d843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:04.426958 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-525bb371-3fc5-4b96-af3e-553b5903d843 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:20:04.428193 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-525bb371-3fc5-4b96-af3e-553b5903d843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:20:04.487228 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1063/2122] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:20:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:20:04.496944 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64168114-f9ff-4829-8118-6c503c829755 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:04.498356 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-64168114-f9ff-4829-8118-6c503c829755 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:20:04.555335 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64168114-f9ff-4829-8118-6c503c829755 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:04.559119 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccc9000-82c7-433a-9306-598e73635170 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:04.559752 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccc9000-82c7-433a-9306-598e73635170 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:04.560033 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccc9000-82c7-433a-9306-598e73635170 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:04.560311 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccc9000-82c7-433a-9306-598e73635170 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:04.569453 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccc9000-82c7-433a-9306-598e73635170 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:04.570880 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccc9000-82c7-433a-9306-598e73635170 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:04.582287 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1060/2123] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:20:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:20:04.848974 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525bb371-3fc5-4b96-af3e-553b5903d843 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:04.848974 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-525bb371-3fc5-4b96-af3e-553b5903d843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:20:04.861658 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525bb371-3fc5-4b96-af3e-553b5903d843 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:04.866861 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c37a2ef-4a5e-4873-8207-81179b159676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:04.867526 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c37a2ef-4a5e-4873-8207-81179b159676 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:04.867906 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c37a2ef-4a5e-4873-8207-81179b159676 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:04.868251 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c37a2ef-4a5e-4873-8207-81179b159676 admin admin] PATH_INFO: `/v3/projects/efed7e85274942b0b62d1c649c9a7619` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:04.869479 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c37a2ef-4a5e-4873-8207-81179b159676 admin admin] RBAC: Authorizing `identity:delete_project(project_id=efed7e85274942b0b62d1c649c9a7619)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:04.871502 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c37a2ef-4a5e-4873-8207-81179b159676 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:04.914491 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:20:04.923010 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-2c37a2ef-4a5e-4873-8207-81179b159676 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'efed7e85274942b0b62d1c649c9a7619', 'request_id': 'req-2c37a2ef-4a5e-4873-8207-81179b159676'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:04.930422 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-2c37a2ef-4a5e-4873-8207-81179b159676 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'efed7e85274942b0b62d1c649c9a7619', 'request_id': 'req-2c37a2ef-4a5e-4873-8207-81179b159676'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:04.936872 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-2c37a2ef-4a5e-4873-8207-81179b159676 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'efed7e85274942b0b62d1c649c9a7619', 'request_id': 'req-2c37a2ef-4a5e-4873-8207-81179b159676'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:04.947866 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-2c37a2ef-4a5e-4873-8207-81179b159676 admin admin] The token cache is being invalidate because project efed7e85274942b0b62d1c649c9a7619 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 19:20:04.948154 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-2c37a2ef-4a5e-4873-8207-81179b159676 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:04.948787 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-2c37a2ef-4a5e-4873-8207-81179b159676 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:04.950267 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1064/2124] 10.222.0.61 () {58 vars in 1233 bytes} [Mon Aug 22 19:20:04 2022] DELETE /identity/v3/projects/efed7e85274942b0b62d1c649c9a7619 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:20:04.954230 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccc9000-82c7-433a-9306-598e73635170 service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:04.964880 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccc9000-82c7-433a-9306-598e73635170 service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:05.007125 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:20:05.026092 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda5aa0a-90ee-4b51-bc4c-30fc493d2f7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:05.026722 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda5aa0a-90ee-4b51-bc4c-30fc493d2f7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:05.027000 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda5aa0a-90ee-4b51-bc4c-30fc493d2f7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:05.027273 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda5aa0a-90ee-4b51-bc4c-30fc493d2f7c admin admin] PATH_INFO: `/v3/users/e6ffd9b417d74c1cbc51a47ef99679f2` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:05.028438 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda5aa0a-90ee-4b51-bc4c-30fc493d2f7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6ffd9b417d74c1cbc51a47ef99679f2)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:05.030344 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda5aa0a-90ee-4b51-bc4c-30fc493d2f7c admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:05.079401 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-eda5aa0a-90ee-4b51-bc4c-30fc493d2f7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6ffd9b417d74c1cbc51a47ef99679f2'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:05.080284 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-eda5aa0a-90ee-4b51-bc4c-30fc493d2f7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6ffd9b417d74c1cbc51a47ef99679f2'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:05.088160 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-eda5aa0a-90ee-4b51-bc4c-30fc493d2f7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6ffd9b417d74c1cbc51a47ef99679f2'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:05.106872 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-eda5aa0a-90ee-4b51-bc4c-30fc493d2f7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6ffd9b417d74c1cbc51a47ef99679f2'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:05.136588 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-eda5aa0a-90ee-4b51-bc4c-30fc493d2f7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6ffd9b417d74c1cbc51a47ef99679f2'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:05.148740 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1061/2125] 10.222.0.61 () {58 vars in 1224 bytes} [Mon Aug 22 19:20:04 2022] DELETE /identity/v3/users/e6ffd9b417d74c1cbc51a47ef99679f2 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:20:05.486009 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c37a2ef-4a5e-4873-8207-81179b159676 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:05.487598 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-2c37a2ef-4a5e-4873-8207-81179b159676 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:20:05.501435 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c37a2ef-4a5e-4873-8207-81179b159676 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:05.560603 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e09f60e-5c98-4a41-87f0-4bd91a466058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:05.561238 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e09f60e-5c98-4a41-87f0-4bd91a466058 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:05.561517 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e09f60e-5c98-4a41-87f0-4bd91a466058 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:05.561788 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e09f60e-5c98-4a41-87f0-4bd91a466058 admin admin] PATH_INFO: `/v3/projects/a78febef45b144148fb43c42bcca5fd0` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:05.562907 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e09f60e-5c98-4a41-87f0-4bd91a466058 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a78febef45b144148fb43c42bcca5fd0)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:05.564853 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e09f60e-5c98-4a41-87f0-4bd91a466058 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:05.614444 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-7e09f60e-5c98-4a41-87f0-4bd91a466058 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a78febef45b144148fb43c42bcca5fd0', 'request_id': 'req-7e09f60e-5c98-4a41-87f0-4bd91a466058'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:05.621506 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-7e09f60e-5c98-4a41-87f0-4bd91a466058 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a78febef45b144148fb43c42bcca5fd0', 'request_id': 'req-7e09f60e-5c98-4a41-87f0-4bd91a466058'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:05.628204 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-7e09f60e-5c98-4a41-87f0-4bd91a466058 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a78febef45b144148fb43c42bcca5fd0', 'request_id': 'req-7e09f60e-5c98-4a41-87f0-4bd91a466058'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:05.638475 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-7e09f60e-5c98-4a41-87f0-4bd91a466058 admin admin] The token cache is being invalidate because project a78febef45b144148fb43c42bcca5fd0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 19:20:05.638869 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-7e09f60e-5c98-4a41-87f0-4bd91a466058 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:05.639502 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-7e09f60e-5c98-4a41-87f0-4bd91a466058 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:05.641015 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1065/2126] 10.222.0.61 () {58 vars in 1233 bytes} [Mon Aug 22 19:20:05 2022] DELETE /identity/v3/projects/a78febef45b144148fb43c42bcca5fd0 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:20:05.646006 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4057c57a-a958-4d18-bb38-09e8549d3fb9 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:05.646478 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4057c57a-a958-4d18-bb38-09e8549d3fb9 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:05.646478 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4057c57a-a958-4d18-bb38-09e8549d3fb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:05.665513 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-4057c57a-a958-4d18-bb38-09e8549d3fb9 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:20:05.667223 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-4057c57a-a958-4d18-bb38-09e8549d3fb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:20:05.730240 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1062/2127] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:20:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:20:05.739009 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d17dee-1a3b-4ca7-9704-e3f77c4eed2e None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:05.739295 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d17dee-1a3b-4ca7-9704-e3f77c4eed2e None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:05.739570 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d17dee-1a3b-4ca7-9704-e3f77c4eed2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:05.759238 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-75d17dee-1a3b-4ca7-9704-e3f77c4eed2e None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:20:05.761132 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-75d17dee-1a3b-4ca7-9704-e3f77c4eed2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:20:05.768898 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1066/2128] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:20:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:20:05.772915 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4057c57a-a958-4d18-bb38-09e8549d3fb9 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:05.783302 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4057c57a-a958-4d18-bb38-09e8549d3fb9 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:05.788086 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c60914-4627-4eb2-bd42-8b1aba2fd0dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:05.788680 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c60914-4627-4eb2-bd42-8b1aba2fd0dc admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:05.788957 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c60914-4627-4eb2-bd42-8b1aba2fd0dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:05.789226 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c60914-4627-4eb2-bd42-8b1aba2fd0dc admin admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:05.789871 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c60914-4627-4eb2-bd42-8b1aba2fd0dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:20:05.790176 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c60914-4627-4eb2-bd42-8b1aba2fd0dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:05.792420 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c60914-4627-4eb2-bd42-8b1aba2fd0dc admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:05.804522 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1063/2129] 10.222.0.61 () {58 vars in 1153 bytes} [Mon Aug 22 19:20:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:20:05.808910 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d17dee-1a3b-4ca7-9704-e3f77c4eed2e None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:05.818681 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d17dee-1a3b-4ca7-9704-e3f77c4eed2e None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:05.823465 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab87918f-525a-44a5-83d3-53d2a78742de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:05.824098 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab87918f-525a-44a5-83d3-53d2a78742de admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:05.824376 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab87918f-525a-44a5-83d3-53d2a78742de admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:05.824646 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab87918f-525a-44a5-83d3-53d2a78742de admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:05.825277 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab87918f-525a-44a5-83d3-53d2a78742de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:05.827167 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab87918f-525a-44a5-83d3-53d2a78742de admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:05.854880 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1067/2130] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 19:20:05 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:20:05.859075 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c60914-4627-4eb2-bd42-8b1aba2fd0dc admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:05.869357 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c60914-4627-4eb2-bd42-8b1aba2fd0dc admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:05.914393 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:20:05.929168 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c8eacb-0d4d-460b-8702-e3c113d7b032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:05.929787 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c8eacb-0d4d-460b-8702-e3c113d7b032 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:05.930093 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c8eacb-0d4d-460b-8702-e3c113d7b032 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:05.930372 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c8eacb-0d4d-460b-8702-e3c113d7b032 admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:05.930988 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c8eacb-0d4d-460b-8702-e3c113d7b032 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:05.932967 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c8eacb-0d4d-460b-8702-e3c113d7b032 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:05.953991 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1064/2131] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 19:20:05 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:20:05.958076 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab87918f-525a-44a5-83d3-53d2a78742de admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:05.969213 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab87918f-525a-44a5-83d3-53d2a78742de admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:05.973879 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15be49d9-f0d4-4b92-9553-586a657cc48a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:05.974472 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15be49d9-f0d4-4b92-9553-586a657cc48a admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:05.974752 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15be49d9-f0d4-4b92-9553-586a657cc48a admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:05.975024 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15be49d9-f0d4-4b92-9553-586a657cc48a admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:05.975690 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15be49d9-f0d4-4b92-9553-586a657cc48a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:05.977538 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15be49d9-f0d4-4b92-9553-586a657cc48a admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:05.987957 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1068/2132] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:20:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 19:20:05.992090 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c8eacb-0d4d-460b-8702-e3c113d7b032 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:06.002316 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c8eacb-0d4d-460b-8702-e3c113d7b032 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:06.006964 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ede2438-210e-4832-a1d3-f4b59f0f0b77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:06.007612 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ede2438-210e-4832-a1d3-f4b59f0f0b77 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:06.007909 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ede2438-210e-4832-a1d3-f4b59f0f0b77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:06.008181 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ede2438-210e-4832-a1d3-f4b59f0f0b77 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:06.008949 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ede2438-210e-4832-a1d3-f4b59f0f0b77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:06.010603 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ede2438-210e-4832-a1d3-f4b59f0f0b77 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:06.021107 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1065/2133] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:20:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:20:06.025718 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15be49d9-f0d4-4b92-9553-586a657cc48a admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:06.035894 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15be49d9-f0d4-4b92-9553-586a657cc48a admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:06.040758 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777ec2ad-aa76-4f23-aac8-f64df73f9c98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:06.041984 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ec2ad-aa76-4f23-aac8-f64df73f9c98 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:06.042262 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ec2ad-aa76-4f23-aac8-f64df73f9c98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:06.042535 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ec2ad-aa76-4f23-aac8-f64df73f9c98 admin admin] PATH_INFO: `/v3/projects/842c8a124ef946b8af1b907e2dec7957/users/aba09ef7cb7646119da853e96aec4083/roles/74a40bc819cb42eea7e966a4b2a016f1` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:06.057665 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777ec2ad-aa76-4f23-aac8-f64df73f9c98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=842c8a124ef946b8af1b907e2dec7957, user_id=aba09ef7cb7646119da853e96aec4083, role_id=74a40bc819cb42eea7e966a4b2a016f1)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:06.060394 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777ec2ad-aa76-4f23-aac8-f64df73f9c98 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:06.062620 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:20:06.074678 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1069/2134] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:20:06 2022] PUT /identity/v3/projects/842c8a124ef946b8af1b907e2dec7957/users/aba09ef7cb7646119da853e96aec4083/roles/74a40bc819cb42eea7e966a4b2a016f1 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:20:06.085123 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d26a034-7aaa-48c1-bdd2-9d4b37284985 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:06.085414 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d26a034-7aaa-48c1-bdd2-9d4b37284985 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:06.085687 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d26a034-7aaa-48c1-bdd2-9d4b37284985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:06.114015 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-7d26a034-7aaa-48c1-bdd2-9d4b37284985 None None] MFA Rules not processed for user `aba09ef7cb7646119da853e96aec4083`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:20:06.115265 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-7d26a034-7aaa-48c1-bdd2-9d4b37284985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:20:06.186852 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1066/2135] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:20:06 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:20:06.198909 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777ec2ad-aa76-4f23-aac8-f64df73f9c98 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:06.200596 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-777ec2ad-aa76-4f23-aac8-f64df73f9c98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:20:06.260470 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777ec2ad-aa76-4f23-aac8-f64df73f9c98 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:06.264175 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ade546-7161-475e-aa4b-53411ba3ddee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e79498873d4ed29a0e19bbea9b9214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:06.264780 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ade546-7161-475e-aa4b-53411ba3ddee service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:06.265054 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ade546-7161-475e-aa4b-53411ba3ddee service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:06.265320 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ade546-7161-475e-aa4b-53411ba3ddee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:06.273747 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ade546-7161-475e-aa4b-53411ba3ddee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:06.276068 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ade546-7161-475e-aa4b-53411ba3ddee service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:06.285146 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1070/2136] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:20:06 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:20:08.957653 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d26a034-7aaa-48c1-bdd2-9d4b37284985 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:08.960904 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-7d26a034-7aaa-48c1-bdd2-9d4b37284985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:20:09.028534 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d26a034-7aaa-48c1-bdd2-9d4b37284985 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:09.033060 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52966954-ff5a-47a3-bd62-23bfce8bbe4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:09.033769 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52966954-ff5a-47a3-bd62-23bfce8bbe4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:09.034042 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52966954-ff5a-47a3-bd62-23bfce8bbe4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:09.034340 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52966954-ff5a-47a3-bd62-23bfce8bbe4b admin admin] PATH_INFO: `/v3/users/a5a9dd005a9d44f79d298ccc70d38bac` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:09.035463 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52966954-ff5a-47a3-bd62-23bfce8bbe4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5a9dd005a9d44f79d298ccc70d38bac)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:09.037408 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52966954-ff5a-47a3-bd62-23bfce8bbe4b admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:09.083410 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-52966954-ff5a-47a3-bd62-23bfce8bbe4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5a9dd005a9d44f79d298ccc70d38bac'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:09.084074 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-52966954-ff5a-47a3-bd62-23bfce8bbe4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5a9dd005a9d44f79d298ccc70d38bac'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:09.091503 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-52966954-ff5a-47a3-bd62-23bfce8bbe4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5a9dd005a9d44f79d298ccc70d38bac'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:09.094249 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:20:09.099787 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-52966954-ff5a-47a3-bd62-23bfce8bbe4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5a9dd005a9d44f79d298ccc70d38bac'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:09.113635 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-52966954-ff5a-47a3-bd62-23bfce8bbe4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5a9dd005a9d44f79d298ccc70d38bac'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:09.119920 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1067/2137] 10.222.0.61 () {58 vars in 1224 bytes} [Mon Aug 22 19:20:08 2022] DELETE /identity/v3/users/a5a9dd005a9d44f79d298ccc70d38bac => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:20:09.124474 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a512ef16-f603-48d7-a8b9-f6d5c537a7c6 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:09.124968 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a512ef16-f603-48d7-a8b9-f6d5c537a7c6 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:09.125047 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a512ef16-f603-48d7-a8b9-f6d5c537a7c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:09.145222 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-a512ef16-f603-48d7-a8b9-f6d5c537a7c6 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:20:09.146457 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-a512ef16-f603-48d7-a8b9-f6d5c537a7c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:20:09.206529 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1071/2138] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:20:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:20:09.216632 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52966954-ff5a-47a3-bd62-23bfce8bbe4b admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:09.218127 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-52966954-ff5a-47a3-bd62-23bfce8bbe4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:20:09.279165 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52966954-ff5a-47a3-bd62-23bfce8bbe4b admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:09.288126 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7ce7aa-7413-439e-9579-c8d01062480d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:09.288734 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7ce7aa-7413-439e-9579-c8d01062480d service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:09.289005 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7ce7aa-7413-439e-9579-c8d01062480d service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:09.289255 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7ce7aa-7413-439e-9579-c8d01062480d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:09.301094 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7ce7aa-7413-439e-9579-c8d01062480d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:09.302568 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7ce7aa-7413-439e-9579-c8d01062480d service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:09.313348 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1068/2139] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:20:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:20:09.595428 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a512ef16-f603-48d7-a8b9-f6d5c537a7c6 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:09.596932 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-a512ef16-f603-48d7-a8b9-f6d5c537a7c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:20:09.612935 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a512ef16-f603-48d7-a8b9-f6d5c537a7c6 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:09.618676 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f781475-7dc6-4e35-add9-00019d4637c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:09.619772 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f781475-7dc6-4e35-add9-00019d4637c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:09.620055 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f781475-7dc6-4e35-add9-00019d4637c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:09.620331 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f781475-7dc6-4e35-add9-00019d4637c5 admin admin] PATH_INFO: `/v3/projects/228376dec9af4deabeef24c9b801a300` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:09.621485 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f781475-7dc6-4e35-add9-00019d4637c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=228376dec9af4deabeef24c9b801a300)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:09.623436 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f781475-7dc6-4e35-add9-00019d4637c5 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:09.670447 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-0f781475-7dc6-4e35-add9-00019d4637c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '228376dec9af4deabeef24c9b801a300', 'request_id': 'req-0f781475-7dc6-4e35-add9-00019d4637c5'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:09.677605 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-0f781475-7dc6-4e35-add9-00019d4637c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '228376dec9af4deabeef24c9b801a300', 'request_id': 'req-0f781475-7dc6-4e35-add9-00019d4637c5'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:09.683901 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-0f781475-7dc6-4e35-add9-00019d4637c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '228376dec9af4deabeef24c9b801a300', 'request_id': 'req-0f781475-7dc6-4e35-add9-00019d4637c5'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:09.694128 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-0f781475-7dc6-4e35-add9-00019d4637c5 admin admin] The token cache is being invalidate because project 228376dec9af4deabeef24c9b801a300 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 19:20:09.694413 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-0f781475-7dc6-4e35-add9-00019d4637c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:09.695080 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-0f781475-7dc6-4e35-add9-00019d4637c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:09.696964 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1072/2140] 10.222.0.61 () {58 vars in 1233 bytes} [Mon Aug 22 19:20:09 2022] DELETE /identity/v3/projects/228376dec9af4deabeef24c9b801a300 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:20:09.701002 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7ce7aa-7413-439e-9579-c8d01062480d service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:09.710871 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7ce7aa-7413-439e-9579-c8d01062480d service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:09.769092 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69172da-13a8-4a72-bd7f-a33678a81953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:09.769633 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69172da-13a8-4a72-bd7f-a33678a81953 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:09.769911 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69172da-13a8-4a72-bd7f-a33678a81953 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:09.770182 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69172da-13a8-4a72-bd7f-a33678a81953 admin admin] PATH_INFO: `/v3/users/93daa36b137348d9b3d91e01c2e2c37a` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:09.771299 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69172da-13a8-4a72-bd7f-a33678a81953 admin admin] RBAC: Authorizing `identity:delete_user(user_id=93daa36b137348d9b3d91e01c2e2c37a)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:09.773250 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69172da-13a8-4a72-bd7f-a33678a81953 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:09.821133 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a69172da-13a8-4a72-bd7f-a33678a81953 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93daa36b137348d9b3d91e01c2e2c37a'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:09.824121 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a69172da-13a8-4a72-bd7f-a33678a81953 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93daa36b137348d9b3d91e01c2e2c37a'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:09.831809 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a69172da-13a8-4a72-bd7f-a33678a81953 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93daa36b137348d9b3d91e01c2e2c37a'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:09.839965 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a69172da-13a8-4a72-bd7f-a33678a81953 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93daa36b137348d9b3d91e01c2e2c37a'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:09.852410 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a69172da-13a8-4a72-bd7f-a33678a81953 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93daa36b137348d9b3d91e01c2e2c37a'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:09.853820 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:20:09.859941 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1069/2141] 10.222.0.61 () {58 vars in 1224 bytes} [Mon Aug 22 19:20:09 2022] DELETE /identity/v3/users/93daa36b137348d9b3d91e01c2e2c37a => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:20:09.991213 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f781475-7dc6-4e35-add9-00019d4637c5 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:09.992723 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-0f781475-7dc6-4e35-add9-00019d4637c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:20:10.016546 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f781475-7dc6-4e35-add9-00019d4637c5 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:10.061415 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:20:10.120049 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89893b5e-8a7b-417d-8934-4a6caf1dc2cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:10.121078 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89893b5e-8a7b-417d-8934-4a6caf1dc2cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:10.121548 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89893b5e-8a7b-417d-8934-4a6caf1dc2cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:10.122021 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89893b5e-8a7b-417d-8934-4a6caf1dc2cb admin admin] PATH_INFO: `/v3/users/aba09ef7cb7646119da853e96aec4083` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:10.123395 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89893b5e-8a7b-417d-8934-4a6caf1dc2cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=aba09ef7cb7646119da853e96aec4083)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:10.127340 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89893b5e-8a7b-417d-8934-4a6caf1dc2cb admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:10.200993 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-89893b5e-8a7b-417d-8934-4a6caf1dc2cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aba09ef7cb7646119da853e96aec4083'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:10.201815 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-89893b5e-8a7b-417d-8934-4a6caf1dc2cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aba09ef7cb7646119da853e96aec4083'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:10.209738 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-89893b5e-8a7b-417d-8934-4a6caf1dc2cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aba09ef7cb7646119da853e96aec4083'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:10.219011 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-89893b5e-8a7b-417d-8934-4a6caf1dc2cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aba09ef7cb7646119da853e96aec4083'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:10.235559 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-89893b5e-8a7b-417d-8934-4a6caf1dc2cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aba09ef7cb7646119da853e96aec4083'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:10.242429 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1073/2142] 10.222.0.61 () {58 vars in 1224 bytes} [Mon Aug 22 19:20:09 2022] DELETE /identity/v3/users/aba09ef7cb7646119da853e96aec4083 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:20:10.246936 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e23e5a7-1631-4f5c-9d81-87083af9f8c2 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:10.247418 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e23e5a7-1631-4f5c-9d81-87083af9f8c2 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:10.247717 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e23e5a7-1631-4f5c-9d81-87083af9f8c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:10.266652 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-2e23e5a7-1631-4f5c-9d81-87083af9f8c2 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:20:10.267993 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-2e23e5a7-1631-4f5c-9d81-87083af9f8c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:20:10.299506 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89893b5e-8a7b-417d-8934-4a6caf1dc2cb admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:10.301222 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-89893b5e-8a7b-417d-8934-4a6caf1dc2cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:20:10.355491 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1070/2143] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:20:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:20:10.361688 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e23e5a7-1631-4f5c-9d81-87083af9f8c2 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:10.363104 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-2e23e5a7-1631-4f5c-9d81-87083af9f8c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:20:10.381761 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89893b5e-8a7b-417d-8934-4a6caf1dc2cb admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:10.386107 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca6fec5-713a-4259-a42c-e48a87116eb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:10.387397 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca6fec5-713a-4259-a42c-e48a87116eb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:10.387711 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca6fec5-713a-4259-a42c-e48a87116eb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:10.388021 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca6fec5-713a-4259-a42c-e48a87116eb5 admin admin] PATH_INFO: `/v3/projects/5163a25801f84b9d864413197da52869` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:10.389161 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca6fec5-713a-4259-a42c-e48a87116eb5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5163a25801f84b9d864413197da52869)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:10.391077 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca6fec5-713a-4259-a42c-e48a87116eb5 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:10.439828 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e23e5a7-1631-4f5c-9d81-87083af9f8c2 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:10.454819 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:20:10.464063 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-dca6fec5-713a-4259-a42c-e48a87116eb5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5163a25801f84b9d864413197da52869', 'request_id': 'req-dca6fec5-713a-4259-a42c-e48a87116eb5'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:10.472828 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-dca6fec5-713a-4259-a42c-e48a87116eb5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5163a25801f84b9d864413197da52869', 'request_id': 'req-dca6fec5-713a-4259-a42c-e48a87116eb5'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:10.480581 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-dca6fec5-713a-4259-a42c-e48a87116eb5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5163a25801f84b9d864413197da52869', 'request_id': 'req-dca6fec5-713a-4259-a42c-e48a87116eb5'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:10.491443 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-dca6fec5-713a-4259-a42c-e48a87116eb5 admin admin] The token cache is being invalidate because project 5163a25801f84b9d864413197da52869 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 19:20:10.491761 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26600a22-7b8a-4b63-8f7b-bd79c24821b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:10.491926 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-dca6fec5-713a-4259-a42c-e48a87116eb5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:10.492380 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26600a22-7b8a-4b63-8f7b-bd79c24821b0 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:10.492536 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-dca6fec5-713a-4259-a42c-e48a87116eb5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:10.492771 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26600a22-7b8a-4b63-8f7b-bd79c24821b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:10.492938 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26600a22-7b8a-4b63-8f7b-bd79c24821b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:10.494512 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1074/2144] 10.222.0.61 () {58 vars in 1233 bytes} [Mon Aug 22 19:20:10 2022] DELETE /identity/v3/projects/5163a25801f84b9d864413197da52869 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:20:10.495613 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-26600a22-7b8a-4b63-8f7b-bd79c24821b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:20:10.498305 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca6fec5-713a-4259-a42c-e48a87116eb5 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:10.499558 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-dca6fec5-713a-4259-a42c-e48a87116eb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:20:10.563024 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26600a22-7b8a-4b63-8f7b-bd79c24821b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:10.564517 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26600a22-7b8a-4b63-8f7b-bd79c24821b0 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:10.569979 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca6fec5-713a-4259-a42c-e48a87116eb5 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:10.574579 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1071/2145] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:20:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 19:20:10.575723 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e14d8b-e58e-4bc6-a832-e59022a78cad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:10.576471 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e14d8b-e58e-4bc6-a832-e59022a78cad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:10.576808 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e14d8b-e58e-4bc6-a832-e59022a78cad admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:10.577123 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e14d8b-e58e-4bc6-a832-e59022a78cad admin admin] PATH_INFO: `/v3/users/981672304b7642b3bd8e9adbc2de29e8` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:10.578451 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e14d8b-e58e-4bc6-a832-e59022a78cad admin admin] RBAC: Authorizing `identity:delete_user(user_id=981672304b7642b3bd8e9adbc2de29e8)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:10.580710 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e14d8b-e58e-4bc6-a832-e59022a78cad admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:10.633127 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-74e14d8b-e58e-4bc6-a832-e59022a78cad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '981672304b7642b3bd8e9adbc2de29e8'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:10.633822 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-74e14d8b-e58e-4bc6-a832-e59022a78cad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '981672304b7642b3bd8e9adbc2de29e8'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:10.641833 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-74e14d8b-e58e-4bc6-a832-e59022a78cad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '981672304b7642b3bd8e9adbc2de29e8'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:10.656940 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-74e14d8b-e58e-4bc6-a832-e59022a78cad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '981672304b7642b3bd8e9adbc2de29e8'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:10.671497 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-74e14d8b-e58e-4bc6-a832-e59022a78cad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '981672304b7642b3bd8e9adbc2de29e8'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:10.678380 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1075/2146] 10.222.0.61 () {58 vars in 1224 bytes} [Mon Aug 22 19:20:10 2022] DELETE /identity/v3/users/981672304b7642b3bd8e9adbc2de29e8 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 19:20:10.917893 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26600a22-7b8a-4b63-8f7b-bd79c24821b0 service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:10.919440 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-26600a22-7b8a-4b63-8f7b-bd79c24821b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:20:10.928262 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:20:10.933710 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26600a22-7b8a-4b63-8f7b-bd79c24821b0 service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:11.017900 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08900fe2-7248-483c-99ff-9cc6390b609b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:11.018803 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08900fe2-7248-483c-99ff-9cc6390b609b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:11.019074 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08900fe2-7248-483c-99ff-9cc6390b609b admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:11.019353 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08900fe2-7248-483c-99ff-9cc6390b609b admin admin] PATH_INFO: `/v3/projects/842c8a124ef946b8af1b907e2dec7957` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:11.020500 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08900fe2-7248-483c-99ff-9cc6390b609b admin admin] RBAC: Authorizing `identity:delete_project(project_id=842c8a124ef946b8af1b907e2dec7957)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:11.022376 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08900fe2-7248-483c-99ff-9cc6390b609b admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:11.040302 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e14d8b-e58e-4bc6-a832-e59022a78cad admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:11.042227 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-74e14d8b-e58e-4bc6-a832-e59022a78cad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:20:11.060140 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e14d8b-e58e-4bc6-a832-e59022a78cad admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:11.068043 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9902bb03-53a4-44c9-ba91-39cf7c91567d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:11.068693 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9902bb03-53a4-44c9-ba91-39cf7c91567d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:11.068972 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9902bb03-53a4-44c9-ba91-39cf7c91567d admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:11.069238 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9902bb03-53a4-44c9-ba91-39cf7c91567d admin admin] PATH_INFO: `/v3/projects/b07bb4a87ad7467cb165c16a4c2f5288` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:11.070371 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9902bb03-53a4-44c9-ba91-39cf7c91567d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b07bb4a87ad7467cb165c16a4c2f5288)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:11.072910 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9902bb03-53a4-44c9-ba91-39cf7c91567d admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:11.077719 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-08900fe2-7248-483c-99ff-9cc6390b609b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '842c8a124ef946b8af1b907e2dec7957', 'request_id': 'req-08900fe2-7248-483c-99ff-9cc6390b609b'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:11.085818 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-08900fe2-7248-483c-99ff-9cc6390b609b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '842c8a124ef946b8af1b907e2dec7957', 'request_id': 'req-08900fe2-7248-483c-99ff-9cc6390b609b'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:11.092830 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-08900fe2-7248-483c-99ff-9cc6390b609b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '842c8a124ef946b8af1b907e2dec7957', 'request_id': 'req-08900fe2-7248-483c-99ff-9cc6390b609b'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:11.111716 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-08900fe2-7248-483c-99ff-9cc6390b609b admin admin] The token cache is being invalidate because project 842c8a124ef946b8af1b907e2dec7957 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 19:20:11.112064 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-08900fe2-7248-483c-99ff-9cc6390b609b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:11.112652 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-08900fe2-7248-483c-99ff-9cc6390b609b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:11.114565 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1072/2147] 10.222.0.61 () {58 vars in 1233 bytes} [Mon Aug 22 19:20:10 2022] DELETE /identity/v3/projects/842c8a124ef946b8af1b907e2dec7957 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 19:20:11.120117 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9745a28-37c1-4cf5-82d9-6f6f73996ae3 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:11.120410 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9745a28-37c1-4cf5-82d9-6f6f73996ae3 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:11.120680 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9745a28-37c1-4cf5-82d9-6f6f73996ae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:11.126044 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:20:11.130403 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-9902bb03-53a4-44c9-ba91-39cf7c91567d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b07bb4a87ad7467cb165c16a4c2f5288', 'request_id': 'req-9902bb03-53a4-44c9-ba91-39cf7c91567d'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:11.139046 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-9902bb03-53a4-44c9-ba91-39cf7c91567d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b07bb4a87ad7467cb165c16a4c2f5288', 'request_id': 'req-9902bb03-53a4-44c9-ba91-39cf7c91567d'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:11.142649 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-f9745a28-37c1-4cf5-82d9-6f6f73996ae3 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:20:11.143900 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-f9745a28-37c1-4cf5-82d9-6f6f73996ae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:20:11.145703 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-9902bb03-53a4-44c9-ba91-39cf7c91567d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b07bb4a87ad7467cb165c16a4c2f5288', 'request_id': 'req-9902bb03-53a4-44c9-ba91-39cf7c91567d'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:11.156573 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-9902bb03-53a4-44c9-ba91-39cf7c91567d admin admin] The token cache is being invalidate because project b07bb4a87ad7467cb165c16a4c2f5288 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 19:20:11.156924 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-9902bb03-53a4-44c9-ba91-39cf7c91567d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:11.157615 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-9902bb03-53a4-44c9-ba91-39cf7c91567d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:11.159289 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1076/2148] 10.222.0.61 () {58 vars in 1233 bytes} [Mon Aug 22 19:20:11 2022] DELETE /identity/v3/projects/b07bb4a87ad7467cb165c16a4c2f5288 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 19:20:11.163496 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9492436-6fcd-4131-9b86-7db9ab5165c1 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:11.163810 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9492436-6fcd-4131-9b86-7db9ab5165c1 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:11.164088 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9492436-6fcd-4131-9b86-7db9ab5165c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:11.184132 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-c9492436-6fcd-4131-9b86-7db9ab5165c1 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:20:11.185435 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-c9492436-6fcd-4131-9b86-7db9ab5165c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:20:11.218179 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1073/2149] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:20:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:20:11.225363 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6fce52-8d15-461d-8224-1725a9558ee6 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:11.225765 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6fce52-8d15-461d-8224-1725a9558ee6 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:11.226040 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6fce52-8d15-461d-8224-1725a9558ee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:11.248510 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-9a6fce52-8d15-461d-8224-1725a9558ee6 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:20:11.249865 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-9a6fce52-8d15-461d-8224-1725a9558ee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:20:11.262076 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1074/2150] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:20:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:20:11.266328 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6fce52-8d15-461d-8224-1725a9558ee6 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:11.266798 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1077/2151] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:20:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 19:20:11.274433 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59999c93-7d9b-4345-8933-32c8ae6dd1d4 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:11.274858 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59999c93-7d9b-4345-8933-32c8ae6dd1d4 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:11.275230 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59999c93-7d9b-4345-8933-32c8ae6dd1d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:11.277448 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6fce52-8d15-461d-8224-1725a9558ee6 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:11.283557 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e869ab7a-0454-4bb1-9b58-c22115953b59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:11.284204 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e869ab7a-0454-4bb1-9b58-c22115953b59 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:11.284523 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e869ab7a-0454-4bb1-9b58-c22115953b59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:11.284853 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e869ab7a-0454-4bb1-9b58-c22115953b59 admin admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:11.285638 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e869ab7a-0454-4bb1-9b58-c22115953b59 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:20:11.285942 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e869ab7a-0454-4bb1-9b58-c22115953b59 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:11.288048 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e869ab7a-0454-4bb1-9b58-c22115953b59 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:11.305183 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-59999c93-7d9b-4345-8933-32c8ae6dd1d4 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:20:11.307112 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-59999c93-7d9b-4345-8933-32c8ae6dd1d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:20:11.321197 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1078/2152] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:20:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 19:20:11.321385 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1075/2153] 10.222.0.61 () {58 vars in 1153 bytes} [Mon Aug 22 19:20:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:20:11.322031 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59999c93-7d9b-4345-8933-32c8ae6dd1d4 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:11.328602 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e869ab7a-0454-4bb1-9b58-c22115953b59 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:11.333934 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59999c93-7d9b-4345-8933-32c8ae6dd1d4 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:11.343546 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e869ab7a-0454-4bb1-9b58-c22115953b59 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:11.348247 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0c1219-867c-459a-b9e2-feca2d0a812b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:11.348610 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25ef112-989b-4a06-b5c7-51c639e83868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:11.348861 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0c1219-867c-459a-b9e2-feca2d0a812b admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:11.349139 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0c1219-867c-459a-b9e2-feca2d0a812b admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:11.349416 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0c1219-867c-459a-b9e2-feca2d0a812b admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:11.349532 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25ef112-989b-4a06-b5c7-51c639e83868 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:11.349875 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25ef112-989b-4a06-b5c7-51c639e83868 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:11.350157 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25ef112-989b-4a06-b5c7-51c639e83868 admin admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:11.350739 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0c1219-867c-459a-b9e2-feca2d0a812b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:11.350841 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25ef112-989b-4a06-b5c7-51c639e83868 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:20:11.351123 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25ef112-989b-4a06-b5c7-51c639e83868 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:11.352939 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0c1219-867c-459a-b9e2-feca2d0a812b admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:11.353046 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25ef112-989b-4a06-b5c7-51c639e83868 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:11.375611 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:20:11.378606 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1079/2154] 10.222.0.61 () {58 vars in 1153 bytes} [Mon Aug 22 19:20:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:20:11.379351 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25ef112-989b-4a06-b5c7-51c639e83868 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:11.397656 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25ef112-989b-4a06-b5c7-51c639e83868 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:11.403842 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1076/2155] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 19:20:11 2022] POST /identity/v3/projects => generated 365 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:20:11.404778 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0c1219-867c-459a-b9e2-feca2d0a812b admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:11.406343 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0977c0-6e56-418e-8999-a562015a9b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:11.406343 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0977c0-6e56-418e-8999-a562015a9b4e admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:11.406649 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0977c0-6e56-418e-8999-a562015a9b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:11.406931 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0977c0-6e56-418e-8999-a562015a9b4e admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:11.407565 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0977c0-6e56-418e-8999-a562015a9b4e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:11.409508 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0977c0-6e56-418e-8999-a562015a9b4e admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:11.452786 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1080/2156] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 19:20:11 2022] POST /identity/v3/projects => generated 391 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:20:11.456528 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0977c0-6e56-418e-8999-a562015a9b4e admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:11.485350 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0c1219-867c-459a-b9e2-feca2d0a812b admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:11.541204 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0977c0-6e56-418e-8999-a562015a9b4e admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:11.545960 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d14ffaf-69e7-4604-8252-68242e973bc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:11.546570 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d14ffaf-69e7-4604-8252-68242e973bc4 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:11.546855 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d14ffaf-69e7-4604-8252-68242e973bc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:11.547134 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d14ffaf-69e7-4604-8252-68242e973bc4 admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:11.547799 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d14ffaf-69e7-4604-8252-68242e973bc4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:11.549725 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d14ffaf-69e7-4604-8252-68242e973bc4 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:11.563867 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0238dc9d-0cba-4e61-8acb-b52f3379f945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:11.564514 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0238dc9d-0cba-4e61-8acb-b52f3379f945 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:11.564826 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0238dc9d-0cba-4e61-8acb-b52f3379f945 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:11.565121 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0238dc9d-0cba-4e61-8acb-b52f3379f945 admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:11.565881 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0238dc9d-0cba-4e61-8acb-b52f3379f945 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:11.567911 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0238dc9d-0cba-4e61-8acb-b52f3379f945 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:11.572959 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1081/2157] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 19:20:11 2022] POST /identity/v3/users => generated 296 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 19:20:11.577195 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d14ffaf-69e7-4604-8252-68242e973bc4 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:11.589240 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d14ffaf-69e7-4604-8252-68242e973bc4 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:11.596520 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1077/2158] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 19:20:11 2022] POST /identity/v3/users => generated 283 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:20:11.596657 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768b9964-2553-4b4b-a0ab-edbc9a3d0f84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:11.596657 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768b9964-2553-4b4b-a0ab-edbc9a3d0f84 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:11.596657 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768b9964-2553-4b4b-a0ab-edbc9a3d0f84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:11.596657 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768b9964-2553-4b4b-a0ab-edbc9a3d0f84 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:11.597020 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768b9964-2553-4b4b-a0ab-edbc9a3d0f84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:11.597531 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768b9964-2553-4b4b-a0ab-edbc9a3d0f84 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:11.598295 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0238dc9d-0cba-4e61-8acb-b52f3379f945 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:11.611988 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1082/2159] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:20:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:20:11.612178 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0238dc9d-0cba-4e61-8acb-b52f3379f945 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:11.613471 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768b9964-2553-4b4b-a0ab-edbc9a3d0f84 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:11.615213 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e51a1b4-bd51-4b2b-b359-4efd0b1e380c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:11.615940 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e51a1b4-bd51-4b2b-b359-4efd0b1e380c admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:11.616252 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e51a1b4-bd51-4b2b-b359-4efd0b1e380c admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:11.616523 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e51a1b4-bd51-4b2b-b359-4efd0b1e380c admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:11.617173 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e51a1b4-bd51-4b2b-b359-4efd0b1e380c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:11.618748 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e51a1b4-bd51-4b2b-b359-4efd0b1e380c admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:11.625125 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768b9964-2553-4b4b-a0ab-edbc9a3d0f84 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:11.632133 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1078/2160] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:20:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 19:20:11.632958 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91c8e88-71b4-458d-b633-821aa2ce8cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:11.633109 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e51a1b4-bd51-4b2b-b359-4efd0b1e380c admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:11.633323 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91c8e88-71b4-458d-b633-821aa2ce8cac admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:11.633607 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91c8e88-71b4-458d-b633-821aa2ce8cac admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:11.633983 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91c8e88-71b4-458d-b633-821aa2ce8cac admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:11.635096 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91c8e88-71b4-458d-b633-821aa2ce8cac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:11.636730 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91c8e88-71b4-458d-b633-821aa2ce8cac admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:11.639459 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:20:11.645388 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e51a1b4-bd51-4b2b-b359-4efd0b1e380c admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:11.651601 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f9ef67-b003-49e8-831a-1a7e77895beb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:11.652276 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f9ef67-b003-49e8-831a-1a7e77895beb admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:11.652626 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f9ef67-b003-49e8-831a-1a7e77895beb admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:11.652812 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f9ef67-b003-49e8-831a-1a7e77895beb admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:11.653795 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f9ef67-b003-49e8-831a-1a7e77895beb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:11.655497 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f9ef67-b003-49e8-831a-1a7e77895beb admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:11.659810 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1083/2161] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:20:11 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:20:11.661091 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91c8e88-71b4-458d-b633-821aa2ce8cac admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:11.671821 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1079/2162] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:20:11 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 19:20:11.672500 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91c8e88-71b4-458d-b633-821aa2ce8cac admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:11.675865 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f9ef67-b003-49e8-831a-1a7e77895beb admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:11.677825 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c411848c-4efe-457b-9e7b-39d05f7b43ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:11.678751 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c411848c-4efe-457b-9e7b-39d05f7b43ad admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:11.679313 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c411848c-4efe-457b-9e7b-39d05f7b43ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:11.679835 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c411848c-4efe-457b-9e7b-39d05f7b43ad admin admin] PATH_INFO: `/v3/projects/fbd2c200181b4e40993f6dbfefdd8551/users/17b5d6212f624e219de746b5a029a4c7/roles/74a40bc819cb42eea7e966a4b2a016f1` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:11.687568 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f9ef67-b003-49e8-831a-1a7e77895beb admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:11.692977 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f694b80-c26f-4fe4-8a71-3a0705acd941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:11.693509 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f694b80-c26f-4fe4-8a71-3a0705acd941 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:11.693793 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f694b80-c26f-4fe4-8a71-3a0705acd941 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:11.694063 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f694b80-c26f-4fe4-8a71-3a0705acd941 admin admin] PATH_INFO: `/v3/projects/1c36c32a540942ccbd0ab0ecbce31a58/users/b628635a98d141539faca7618db4b289/roles/74a40bc819cb42eea7e966a4b2a016f1` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:11.696355 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c411848c-4efe-457b-9e7b-39d05f7b43ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbd2c200181b4e40993f6dbfefdd8551, user_id=17b5d6212f624e219de746b5a029a4c7, role_id=74a40bc819cb42eea7e966a4b2a016f1)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:11.699916 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c411848c-4efe-457b-9e7b-39d05f7b43ad admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:11.714629 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f694b80-c26f-4fe4-8a71-3a0705acd941 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c36c32a540942ccbd0ab0ecbce31a58, user_id=b628635a98d141539faca7618db4b289, role_id=74a40bc819cb42eea7e966a4b2a016f1)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:11.717058 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1084/2163] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:20:11 2022] PUT /identity/v3/projects/fbd2c200181b4e40993f6dbfefdd8551/users/17b5d6212f624e219de746b5a029a4c7/roles/74a40bc819cb42eea7e966a4b2a016f1 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:20:11.720571 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f694b80-c26f-4fe4-8a71-3a0705acd941 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:11.727572 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c14feb-4fae-4a3d-b070-3ad5d2616766 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:11.727871 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c14feb-4fae-4a3d-b070-3ad5d2616766 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:11.728237 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c14feb-4fae-4a3d-b070-3ad5d2616766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:11.746567 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1080/2164] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:20:11 2022] PUT /identity/v3/projects/1c36c32a540942ccbd0ab0ecbce31a58/users/b628635a98d141539faca7618db4b289/roles/74a40bc819cb42eea7e966a4b2a016f1 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 19:20:11.757943 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894ebadb-341a-4ba6-b747-4f5592ecf37e None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:11.758230 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894ebadb-341a-4ba6-b747-4f5592ecf37e None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:11.758504 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894ebadb-341a-4ba6-b747-4f5592ecf37e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:11.767166 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-82c14feb-4fae-4a3d-b070-3ad5d2616766 None None] MFA Rules not processed for user `17b5d6212f624e219de746b5a029a4c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:20:11.768017 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-82c14feb-4fae-4a3d-b070-3ad5d2616766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:20:11.790987 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-894ebadb-341a-4ba6-b747-4f5592ecf37e None None] MFA Rules not processed for user `b628635a98d141539faca7618db4b289`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:20:11.792631 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-894ebadb-341a-4ba6-b747-4f5592ecf37e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:20:11.835829 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:20:11.857530 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1085/2165] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:20:11 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 19:20:11.870207 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c14feb-4fae-4a3d-b070-3ad5d2616766 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:11.871877 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-82c14feb-4fae-4a3d-b070-3ad5d2616766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:20:11.886365 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1081/2166] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:20:11 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 19:20:11.898359 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894ebadb-341a-4ba6-b747-4f5592ecf37e None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:11.899803 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-894ebadb-341a-4ba6-b747-4f5592ecf37e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:20:11.949376 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c14feb-4fae-4a3d-b070-3ad5d2616766 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:11.954470 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700260a4-4f76-4bbc-9622-a00acca4f2ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e79498873d4ed29a0e19bbea9b9214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:11.955105 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700260a4-4f76-4bbc-9622-a00acca4f2ce service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:11.955379 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700260a4-4f76-4bbc-9622-a00acca4f2ce service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:11.955683 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700260a4-4f76-4bbc-9622-a00acca4f2ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:11.965871 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700260a4-4f76-4bbc-9622-a00acca4f2ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:11.967575 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700260a4-4f76-4bbc-9622-a00acca4f2ce service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:11.974955 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894ebadb-341a-4ba6-b747-4f5592ecf37e None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:11.979605 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab730877-76f8-45e7-a7f3-a5b11f7366e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e79498873d4ed29a0e19bbea9b9214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:11.980217 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1086/2167] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:20:11 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 19:20:11.980331 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab730877-76f8-45e7-a7f3-a5b11f7366e6 service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:11.980523 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab730877-76f8-45e7-a7f3-a5b11f7366e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:11.980802 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab730877-76f8-45e7-a7f3-a5b11f7366e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:11.989841 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab730877-76f8-45e7-a7f3-a5b11f7366e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:11.991268 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab730877-76f8-45e7-a7f3-a5b11f7366e6 service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:12.003526 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1082/2168] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:20:11 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 19:20:21.976129 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=69,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:20:21.997519 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=38 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:20:37.733526 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747a79a5-e92c-42bf-abd7-b5828c01f5ac None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:37.734068 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747a79a5-e92c-42bf-abd7-b5828c01f5ac None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:37.734068 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747a79a5-e92c-42bf-abd7-b5828c01f5ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:37.760623 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-747a79a5-e92c-42bf-abd7-b5828c01f5ac None None] MFA Rules not processed for user `399396a8fe254da199bc48b0ca5243d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:20:37.762373 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-747a79a5-e92c-42bf-abd7-b5828c01f5ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:20:37.821404 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1087/2169] 10.222.0.61 () {58 vars in 1009 bytes} [Mon Aug 22 19:20:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:20:37.863201 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab730877-76f8-45e7-a7f3-a5b11f7366e6 service cinder] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:37.864647 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-ab730877-76f8-45e7-a7f3-a5b11f7366e6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:20:37.935527 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab730877-76f8-45e7-a7f3-a5b11f7366e6 service cinder] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:37.940503 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cce66d-9302-4c56-8b86-fce5b849998c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2824359491934d0dacaa21a718becee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:37.941100 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cce66d-9302-4c56-8b86-fce5b849998c service swift] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:37.941375 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cce66d-9302-4c56-8b86-fce5b849998c service swift] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:37.941645 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cce66d-9302-4c56-8b86-fce5b849998c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:37.949491 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cce66d-9302-4c56-8b86-fce5b849998c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:37.951057 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cce66d-9302-4c56-8b86-fce5b849998c service swift] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:37.959013 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1083/2170] 10.222.0.61 () {60 vars in 1384 bytes} [Mon Aug 22 19:20:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 22 19:20:38.536817 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f76271c-1094-45f8-89d8-8b373b1819f6 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:38.537108 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f76271c-1094-45f8-89d8-8b373b1819f6 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:38.537385 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f76271c-1094-45f8-89d8-8b373b1819f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:38.557610 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-7f76271c-1094-45f8-89d8-8b373b1819f6 None None] MFA Rules not processed for user `399396a8fe254da199bc48b0ca5243d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:20:38.559619 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-7f76271c-1094-45f8-89d8-8b373b1819f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:20:38.567668 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1088/2171] 10.222.0.61 () {58 vars in 967 bytes} [Mon Aug 22 19:20:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:20:38.581573 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cce66d-9302-4c56-8b86-fce5b849998c service swift] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:38.596496 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cce66d-9302-4c56-8b86-fce5b849998c service swift] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:38.602277 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988c1f47-e280-4d30-ba11-89673fd0279c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2824359491934d0dacaa21a718becee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:38.602877 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988c1f47-e280-4d30-ba11-89673fd0279c service swift] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:38.603152 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988c1f47-e280-4d30-ba11-89673fd0279c service swift] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:38.603429 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988c1f47-e280-4d30-ba11-89673fd0279c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:38.612451 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988c1f47-e280-4d30-ba11-89673fd0279c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:38.613951 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988c1f47-e280-4d30-ba11-89673fd0279c service swift] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:38.625775 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1084/2172] 10.222.0.61 () {60 vars in 1384 bytes} [Mon Aug 22 19:20:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 22 19:20:38.913883 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eb2086-b9a9-40e7-85ad-4e0622f7954f None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:38.914376 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eb2086-b9a9-40e7-85ad-4e0622f7954f None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:38.914499 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eb2086-b9a9-40e7-85ad-4e0622f7954f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:38.937215 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-d0eb2086-b9a9-40e7-85ad-4e0622f7954f None None] MFA Rules not processed for user `399396a8fe254da199bc48b0ca5243d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:20:38.938708 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-d0eb2086-b9a9-40e7-85ad-4e0622f7954f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:20:38.946742 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1089/2173] 10.222.0.61 () {58 vars in 1009 bytes} [Mon Aug 22 19:20:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:20:38.961478 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988c1f47-e280-4d30-ba11-89673fd0279c service swift] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:38.974170 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988c1f47-e280-4d30-ba11-89673fd0279c service swift] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:38.982094 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ee8cbd-83e9-4ccb-b3a2-11436b90af8f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2824359491934d0dacaa21a718becee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:38.983100 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ee8cbd-83e9-4ccb-b3a2-11436b90af8f service swift] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:38.983618 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ee8cbd-83e9-4ccb-b3a2-11436b90af8f service swift] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:38.984062 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ee8cbd-83e9-4ccb-b3a2-11436b90af8f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:38.994834 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ee8cbd-83e9-4ccb-b3a2-11436b90af8f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:38.996650 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ee8cbd-83e9-4ccb-b3a2-11436b90af8f service swift] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:39.005920 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1085/2174] 10.222.0.61 () {60 vars in 1384 bytes} [Mon Aug 22 19:20:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 22 19:20:39.403137 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37698809-1948-4250-be86-6ae1288f7309 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:39.403420 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37698809-1948-4250-be86-6ae1288f7309 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:39.403732 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37698809-1948-4250-be86-6ae1288f7309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:39.422475 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-37698809-1948-4250-be86-6ae1288f7309 None None] MFA Rules not processed for user `399396a8fe254da199bc48b0ca5243d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:20:39.423728 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-37698809-1948-4250-be86-6ae1288f7309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:20:39.431046 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1090/2175] 10.222.0.61 () {58 vars in 967 bytes} [Mon Aug 22 19:20:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:20:39.445143 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ee8cbd-83e9-4ccb-b3a2-11436b90af8f service swift] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:39.459082 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ee8cbd-83e9-4ccb-b3a2-11436b90af8f service swift] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:39.464296 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0638a3-13c1-419d-b7c5-ef4197501947 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2824359491934d0dacaa21a718becee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:39.464917 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0638a3-13c1-419d-b7c5-ef4197501947 service swift] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:39.465726 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0638a3-13c1-419d-b7c5-ef4197501947 service swift] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:39.466000 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0638a3-13c1-419d-b7c5-ef4197501947 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:39.476989 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0638a3-13c1-419d-b7c5-ef4197501947 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:39.480160 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0638a3-13c1-419d-b7c5-ef4197501947 service swift] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:39.490391 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1086/2176] 10.222.0.61 () {60 vars in 1384 bytes} [Mon Aug 22 19:20:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 22 19:20:42.250839 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37698809-1948-4250-be86-6ae1288f7309 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:42.252666 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-37698809-1948-4250-be86-6ae1288f7309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:20:42.329897 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37698809-1948-4250-be86-6ae1288f7309 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:42.334473 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e3041e-0d34-462d-89d3-37e2b5ff3ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:42.335112 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e3041e-0d34-462d-89d3-37e2b5ff3ce6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:42.335389 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e3041e-0d34-462d-89d3-37e2b5ff3ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:42.335684 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e3041e-0d34-462d-89d3-37e2b5ff3ce6 admin admin] PATH_INFO: `/v3/users/942c420f56f5449fbae8b4ff780d1834` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:42.336854 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e3041e-0d34-462d-89d3-37e2b5ff3ce6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=942c420f56f5449fbae8b4ff780d1834)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:42.338792 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e3041e-0d34-462d-89d3-37e2b5ff3ce6 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:42.385053 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-27e3041e-0d34-462d-89d3-37e2b5ff3ce6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '942c420f56f5449fbae8b4ff780d1834'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:42.385716 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-27e3041e-0d34-462d-89d3-37e2b5ff3ce6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '942c420f56f5449fbae8b4ff780d1834'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:42.393191 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-27e3041e-0d34-462d-89d3-37e2b5ff3ce6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '942c420f56f5449fbae8b4ff780d1834'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:42.399821 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-27e3041e-0d34-462d-89d3-37e2b5ff3ce6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '942c420f56f5449fbae8b4ff780d1834'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:42.411754 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-27e3041e-0d34-462d-89d3-37e2b5ff3ce6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '942c420f56f5449fbae8b4ff780d1834'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:42.417644 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1091/2177] 10.222.0.61 () {58 vars in 1224 bytes} [Mon Aug 22 19:20:42 2022] DELETE /identity/v3/users/942c420f56f5449fbae8b4ff780d1834 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 19:20:42.587348 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0638a3-13c1-419d-b7c5-ef4197501947 service swift] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:42.599886 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0638a3-13c1-419d-b7c5-ef4197501947 service swift] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:42.662315 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c38083-8641-4835-b09d-45ff3c5aa283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:42.662960 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c38083-8641-4835-b09d-45ff3c5aa283 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:42.663239 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c38083-8641-4835-b09d-45ff3c5aa283 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:42.663515 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c38083-8641-4835-b09d-45ff3c5aa283 admin admin] PATH_INFO: `/v3/projects/ae986c6b484740a9a4629648d1b65e51` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:42.664726 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c38083-8641-4835-b09d-45ff3c5aa283 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae986c6b484740a9a4629648d1b65e51)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:42.666665 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c38083-8641-4835-b09d-45ff3c5aa283 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:42.716222 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-18c38083-8641-4835-b09d-45ff3c5aa283 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae986c6b484740a9a4629648d1b65e51', 'request_id': 'req-18c38083-8641-4835-b09d-45ff3c5aa283'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:42.718808 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:20:42.724452 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-18c38083-8641-4835-b09d-45ff3c5aa283 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae986c6b484740a9a4629648d1b65e51', 'request_id': 'req-18c38083-8641-4835-b09d-45ff3c5aa283'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:42.733022 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-18c38083-8641-4835-b09d-45ff3c5aa283 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae986c6b484740a9a4629648d1b65e51', 'request_id': 'req-18c38083-8641-4835-b09d-45ff3c5aa283'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:42.744561 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-18c38083-8641-4835-b09d-45ff3c5aa283 admin admin] The token cache is being invalidate because project ae986c6b484740a9a4629648d1b65e51 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 19:20:42.745225 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-18c38083-8641-4835-b09d-45ff3c5aa283 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:42.746546 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-18c38083-8641-4835-b09d-45ff3c5aa283 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:42.749864 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1087/2178] 10.222.0.61 () {58 vars in 1233 bytes} [Mon Aug 22 19:20:42 2022] DELETE /identity/v3/projects/ae986c6b484740a9a4629648d1b65e51 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:20:42.755872 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ddad84-b24c-4499-8eb4-da97afeaf66e None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:42.756164 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ddad84-b24c-4499-8eb4-da97afeaf66e None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:42.756996 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ddad84-b24c-4499-8eb4-da97afeaf66e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:42.768474 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:20:42.781451 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-17ddad84-b24c-4499-8eb4-da97afeaf66e None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:20:42.783205 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-17ddad84-b24c-4499-8eb4-da97afeaf66e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:20:42.840959 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1092/2179] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:20:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:20:42.848712 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fda038-bc61-43be-88d0-15864de03a6f None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:42.848996 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fda038-bc61-43be-88d0-15864de03a6f None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:42.849270 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fda038-bc61-43be-88d0-15864de03a6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:42.869630 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-91fda038-bc61-43be-88d0-15864de03a6f None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:20:42.871851 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-91fda038-bc61-43be-88d0-15864de03a6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:20:42.879899 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1088/2180] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:20:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:20:42.884182 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ddad84-b24c-4499-8eb4-da97afeaf66e None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:42.893961 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ddad84-b24c-4499-8eb4-da97afeaf66e None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:42.898547 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4e2a75-53d9-4fcd-9345-a0bc85cf1cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:42.899148 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4e2a75-53d9-4fcd-9345-a0bc85cf1cc0 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:42.899424 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4e2a75-53d9-4fcd-9345-a0bc85cf1cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:42.899741 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4e2a75-53d9-4fcd-9345-a0bc85cf1cc0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:42.900396 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4e2a75-53d9-4fcd-9345-a0bc85cf1cc0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:20:42.900696 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4e2a75-53d9-4fcd-9345-a0bc85cf1cc0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:42.902385 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4e2a75-53d9-4fcd-9345-a0bc85cf1cc0 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:42.914179 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1093/2181] 10.222.0.61 () {58 vars in 1153 bytes} [Mon Aug 22 19:20:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:20:42.918743 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fda038-bc61-43be-88d0-15864de03a6f None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:42.928267 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fda038-bc61-43be-88d0-15864de03a6f None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:42.933224 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9882dd1-7131-456a-9d7d-31d6535f5e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:42.934515 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9882dd1-7131-456a-9d7d-31d6535f5e41 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:42.934812 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9882dd1-7131-456a-9d7d-31d6535f5e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:42.935082 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9882dd1-7131-456a-9d7d-31d6535f5e41 admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:42.935814 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9882dd1-7131-456a-9d7d-31d6535f5e41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:42.937701 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9882dd1-7131-456a-9d7d-31d6535f5e41 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:42.965453 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1089/2182] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 19:20:42 2022] POST /identity/v3/projects => generated 379 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:20:42.969576 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4e2a75-53d9-4fcd-9345-a0bc85cf1cc0 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:42.979841 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4e2a75-53d9-4fcd-9345-a0bc85cf1cc0 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:43.035331 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06910c85-809d-4feb-a23b-878ed00cc689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:43.035972 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06910c85-809d-4feb-a23b-878ed00cc689 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:43.036252 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06910c85-809d-4feb-a23b-878ed00cc689 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:43.036528 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06910c85-809d-4feb-a23b-878ed00cc689 admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:43.037153 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06910c85-809d-4feb-a23b-878ed00cc689 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:43.039069 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06910c85-809d-4feb-a23b-878ed00cc689 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:43.062704 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1094/2183] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 19:20:42 2022] POST /identity/v3/users => generated 290 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:20:43.066930 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9882dd1-7131-456a-9d7d-31d6535f5e41 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:43.078211 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9882dd1-7131-456a-9d7d-31d6535f5e41 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:43.082946 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd84d7ea-917e-468c-95d6-98e9db57566a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:43.083546 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd84d7ea-917e-468c-95d6-98e9db57566a admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:43.083865 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd84d7ea-917e-468c-95d6-98e9db57566a admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:43.084134 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd84d7ea-917e-468c-95d6-98e9db57566a admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:43.084798 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd84d7ea-917e-468c-95d6-98e9db57566a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:43.086519 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd84d7ea-917e-468c-95d6-98e9db57566a admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:43.097081 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1090/2184] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:20:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:20:43.101255 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06910c85-809d-4feb-a23b-878ed00cc689 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:43.112304 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06910c85-809d-4feb-a23b-878ed00cc689 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:43.117167 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3a0b95-a930-4f5c-bda7-38416b37e8c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:43.118452 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3a0b95-a930-4f5c-bda7-38416b37e8c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:43.118736 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3a0b95-a930-4f5c-bda7-38416b37e8c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:43.119011 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3a0b95-a930-4f5c-bda7-38416b37e8c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:43.119727 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3a0b95-a930-4f5c-bda7-38416b37e8c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:43.121944 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3a0b95-a930-4f5c-bda7-38416b37e8c4 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:43.134839 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1095/2185] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:20:43 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:20:43.139124 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd84d7ea-917e-468c-95d6-98e9db57566a admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:43.150171 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd84d7ea-917e-468c-95d6-98e9db57566a admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:43.155384 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41302fe-39e3-43e3-bf81-0553b02f2d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:43.156096 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41302fe-39e3-43e3-bf81-0553b02f2d2f admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:43.156373 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41302fe-39e3-43e3-bf81-0553b02f2d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:43.156684 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41302fe-39e3-43e3-bf81-0553b02f2d2f admin admin] PATH_INFO: `/v3/projects/19936226466c4cb2869ffbc8e2157825/users/4bb5d58c1c2c47d789486609992a77f5/roles/74a40bc819cb42eea7e966a4b2a016f1` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:43.173162 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41302fe-39e3-43e3-bf81-0553b02f2d2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=19936226466c4cb2869ffbc8e2157825, user_id=4bb5d58c1c2c47d789486609992a77f5, role_id=74a40bc819cb42eea7e966a4b2a016f1)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:43.175860 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41302fe-39e3-43e3-bf81-0553b02f2d2f admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:43.190613 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1091/2186] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:20:43 2022] PUT /identity/v3/projects/19936226466c4cb2869ffbc8e2157825/users/4bb5d58c1c2c47d789486609992a77f5/roles/74a40bc819cb42eea7e966a4b2a016f1 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:20:43.202235 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe38cb6f-468f-4f3c-ae6e-c77033dec7a5 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:43.202646 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe38cb6f-468f-4f3c-ae6e-c77033dec7a5 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:43.203003 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe38cb6f-468f-4f3c-ae6e-c77033dec7a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:43.232972 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-fe38cb6f-468f-4f3c-ae6e-c77033dec7a5 None None] MFA Rules not processed for user `4bb5d58c1c2c47d789486609992a77f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:20:43.234461 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-fe38cb6f-468f-4f3c-ae6e-c77033dec7a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:20:43.297664 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:20:43.301886 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1096/2187] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:20:43 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:20:43.312396 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41302fe-39e3-43e3-bf81-0553b02f2d2f admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:43.314123 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-f41302fe-39e3-43e3-bf81-0553b02f2d2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:20:43.389880 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41302fe-39e3-43e3-bf81-0553b02f2d2f admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:43.395482 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a096fa40-6d1d-4fb4-8ef3-37cf9998bd68 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f43e2756b746a6825ca1a06b7d7c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:43.396116 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a096fa40-6d1d-4fb4-8ef3-37cf9998bd68 service nova] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:43.396396 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a096fa40-6d1d-4fb4-8ef3-37cf9998bd68 service nova] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:43.396673 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a096fa40-6d1d-4fb4-8ef3-37cf9998bd68 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:43.404756 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a096fa40-6d1d-4fb4-8ef3-37cf9998bd68 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:43.406325 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a096fa40-6d1d-4fb4-8ef3-37cf9998bd68 service nova] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:43.414966 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1092/2188] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:20:43 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:20:46.964695 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe38cb6f-468f-4f3c-ae6e-c77033dec7a5 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:46.966156 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-fe38cb6f-468f-4f3c-ae6e-c77033dec7a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:20:47.033437 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe38cb6f-468f-4f3c-ae6e-c77033dec7a5 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:47.038259 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ffac08-2fb3-4130-aeb5-404e7bfc3437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:47.039169 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ffac08-2fb3-4130-aeb5-404e7bfc3437 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:47.039169 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ffac08-2fb3-4130-aeb5-404e7bfc3437 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:47.039420 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ffac08-2fb3-4130-aeb5-404e7bfc3437 admin admin] PATH_INFO: `/v3/users/01a14c36af46451884400a5b8db1702d` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:47.040558 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ffac08-2fb3-4130-aeb5-404e7bfc3437 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01a14c36af46451884400a5b8db1702d)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:47.042458 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ffac08-2fb3-4130-aeb5-404e7bfc3437 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:47.095895 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a096fa40-6d1d-4fb4-8ef3-37cf9998bd68 service nova] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:47.097382 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-a096fa40-6d1d-4fb4-8ef3-37cf9998bd68 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:20:47.102755 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-17ffac08-2fb3-4130-aeb5-404e7bfc3437 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01a14c36af46451884400a5b8db1702d'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:47.103493 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-17ffac08-2fb3-4130-aeb5-404e7bfc3437 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01a14c36af46451884400a5b8db1702d'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:47.114101 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a096fa40-6d1d-4fb4-8ef3-37cf9998bd68 service nova] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:47.118070 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-17ffac08-2fb3-4130-aeb5-404e7bfc3437 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01a14c36af46451884400a5b8db1702d'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:47.120101 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c37b396-6398-48f1-b11d-65f56deb25fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:47.123373 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c37b396-6398-48f1-b11d-65f56deb25fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:47.123373 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c37b396-6398-48f1-b11d-65f56deb25fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:47.123373 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c37b396-6398-48f1-b11d-65f56deb25fd admin admin] PATH_INFO: `/v3/users/17b5d6212f624e219de746b5a029a4c7` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:47.123825 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c37b396-6398-48f1-b11d-65f56deb25fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=17b5d6212f624e219de746b5a029a4c7)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:47.126451 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c37b396-6398-48f1-b11d-65f56deb25fd admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:47.141008 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-17ffac08-2fb3-4130-aeb5-404e7bfc3437 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01a14c36af46451884400a5b8db1702d'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:47.160461 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-17ffac08-2fb3-4130-aeb5-404e7bfc3437 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01a14c36af46451884400a5b8db1702d'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:47.170558 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1097/2189] 10.222.0.61 () {58 vars in 1224 bytes} [Mon Aug 22 19:20:46 2022] DELETE /identity/v3/users/01a14c36af46451884400a5b8db1702d => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:20:47.175336 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dc2e91-fa6d-42f3-bd53-1dd8bd525b3a None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:47.175614 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dc2e91-fa6d-42f3-bd53-1dd8bd525b3a None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:47.175919 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dc2e91-fa6d-42f3-bd53-1dd8bd525b3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:47.199015 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-d6dc2e91-fa6d-42f3-bd53-1dd8bd525b3a None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:20:47.200871 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-d6dc2e91-fa6d-42f3-bd53-1dd8bd525b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:20:47.210314 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-9c37b396-6398-48f1-b11d-65f56deb25fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17b5d6212f624e219de746b5a029a4c7'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:47.211592 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-9c37b396-6398-48f1-b11d-65f56deb25fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17b5d6212f624e219de746b5a029a4c7'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:47.220405 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-9c37b396-6398-48f1-b11d-65f56deb25fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17b5d6212f624e219de746b5a029a4c7'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:47.222967 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:20:47.232314 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-9c37b396-6398-48f1-b11d-65f56deb25fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17b5d6212f624e219de746b5a029a4c7'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:47.252075 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-9c37b396-6398-48f1-b11d-65f56deb25fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17b5d6212f624e219de746b5a029a4c7'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:47.258658 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1093/2190] 10.222.0.61 () {58 vars in 1224 bytes} [Mon Aug 22 19:20:47 2022] DELETE /identity/v3/users/17b5d6212f624e219de746b5a029a4c7 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 19:20:47.264289 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfc70fc-68eb-43c9-ab78-8803f599aa29 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:47.266439 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfc70fc-68eb-43c9-ab78-8803f599aa29 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:47.267232 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfc70fc-68eb-43c9-ab78-8803f599aa29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:47.275022 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1098/2191] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:20:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:20:47.286553 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dc2e91-fa6d-42f3-bd53-1dd8bd525b3a None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:47.292540 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-d6dc2e91-fa6d-42f3-bd53-1dd8bd525b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:20:47.297998 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-7dfc70fc-68eb-43c9-ab78-8803f599aa29 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:20:47.299552 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-7dfc70fc-68eb-43c9-ab78-8803f599aa29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:20:47.319777 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:20:47.361673 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dc2e91-fa6d-42f3-bd53-1dd8bd525b3a None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:47.369778 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1094/2192] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:20:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 19:20:47.380289 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfc70fc-68eb-43c9-ab78-8803f599aa29 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:47.412144 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3691ce83-49f4-4c2d-9ec7-df7b9cebeb25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:47.413288 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3691ce83-49f4-4c2d-9ec7-df7b9cebeb25 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:47.413288 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3691ce83-49f4-4c2d-9ec7-df7b9cebeb25 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:47.413525 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3691ce83-49f4-4c2d-9ec7-df7b9cebeb25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:47.423911 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3691ce83-49f4-4c2d-9ec7-df7b9cebeb25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:47.425348 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3691ce83-49f4-4c2d-9ec7-df7b9cebeb25 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:47.436686 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1099/2193] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:20:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:20:47.447972 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfc70fc-68eb-43c9-ab78-8803f599aa29 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:47.453134 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2dd60e-fe2a-4089-a52d-8bbd45826614 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:47.453699 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2dd60e-fe2a-4089-a52d-8bbd45826614 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:47.454001 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2dd60e-fe2a-4089-a52d-8bbd45826614 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:47.454277 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2dd60e-fe2a-4089-a52d-8bbd45826614 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:47.472789 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2dd60e-fe2a-4089-a52d-8bbd45826614 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:47.474298 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2dd60e-fe2a-4089-a52d-8bbd45826614 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:47.484988 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1095/2194] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:20:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:20:47.637804 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3691ce83-49f4-4c2d-9ec7-df7b9cebeb25 service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:47.639950 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-3691ce83-49f4-4c2d-9ec7-df7b9cebeb25 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:20:47.660165 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3691ce83-49f4-4c2d-9ec7-df7b9cebeb25 service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:47.665646 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a1884e-c58a-40d3-9627-731e2a7c300a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:47.666232 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a1884e-c58a-40d3-9627-731e2a7c300a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:47.666569 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a1884e-c58a-40d3-9627-731e2a7c300a admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:47.666827 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a1884e-c58a-40d3-9627-731e2a7c300a admin admin] PATH_INFO: `/v3/projects/1e9f4c09721e4206869bbc0e49f9ab50` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:47.669188 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a1884e-c58a-40d3-9627-731e2a7c300a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e9f4c09721e4206869bbc0e49f9ab50)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:47.670930 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a1884e-c58a-40d3-9627-731e2a7c300a admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:47.742412 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-c7a1884e-c58a-40d3-9627-731e2a7c300a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e9f4c09721e4206869bbc0e49f9ab50', 'request_id': 'req-c7a1884e-c58a-40d3-9627-731e2a7c300a'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:47.763991 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-c7a1884e-c58a-40d3-9627-731e2a7c300a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e9f4c09721e4206869bbc0e49f9ab50', 'request_id': 'req-c7a1884e-c58a-40d3-9627-731e2a7c300a'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:47.771746 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-c7a1884e-c58a-40d3-9627-731e2a7c300a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e9f4c09721e4206869bbc0e49f9ab50', 'request_id': 'req-c7a1884e-c58a-40d3-9627-731e2a7c300a'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:47.791559 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-c7a1884e-c58a-40d3-9627-731e2a7c300a admin admin] The token cache is being invalidate because project 1e9f4c09721e4206869bbc0e49f9ab50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 19:20:47.791886 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-c7a1884e-c58a-40d3-9627-731e2a7c300a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:47.792661 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-c7a1884e-c58a-40d3-9627-731e2a7c300a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:47.795391 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1100/2195] 10.222.0.61 () {58 vars in 1233 bytes} [Mon Aug 22 19:20:47 2022] DELETE /identity/v3/projects/1e9f4c09721e4206869bbc0e49f9ab50 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:20:47.801356 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0379c2e9-7b60-452c-acd0-475be8107b83 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:47.801810 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a1884e-c58a-40d3-9627-731e2a7c300a admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:47.802043 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0379c2e9-7b60-452c-acd0-475be8107b83 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:47.802357 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0379c2e9-7b60-452c-acd0-475be8107b83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:47.807015 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-c7a1884e-c58a-40d3-9627-731e2a7c300a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:20:47.824206 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-0379c2e9-7b60-452c-acd0-475be8107b83 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:20:47.825424 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-0379c2e9-7b60-452c-acd0-475be8107b83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:20:47.896712 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a1884e-c58a-40d3-9627-731e2a7c300a admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:47.901706 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57648b27-88f4-42ac-aa30-6befc83b47d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:47.902830 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57648b27-88f4-42ac-aa30-6befc83b47d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:47.903102 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57648b27-88f4-42ac-aa30-6befc83b47d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:47.903368 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57648b27-88f4-42ac-aa30-6befc83b47d6 admin admin] PATH_INFO: `/v3/projects/fbd2c200181b4e40993f6dbfefdd8551` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:47.904532 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57648b27-88f4-42ac-aa30-6befc83b47d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fbd2c200181b4e40993f6dbfefdd8551)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:47.906491 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57648b27-88f4-42ac-aa30-6befc83b47d6 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:47.907789 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1096/2196] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:20:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:20:47.909471 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:20:47.911037 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0b9866-0625-4c1b-be6c-f009ddfba234 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:47.911512 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0b9866-0625-4c1b-be6c-f009ddfba234 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:47.911761 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0b9866-0625-4c1b-be6c-f009ddfba234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:47.937771 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-1d0b9866-0625-4c1b-be6c-f009ddfba234 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:20:47.938969 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-1d0b9866-0625-4c1b-be6c-f009ddfba234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:20:47.948095 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1097/2197] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:20:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:20:47.951886 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0b9866-0625-4c1b-be6c-f009ddfba234 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:47.960843 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0b9866-0625-4c1b-be6c-f009ddfba234 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:47.969581 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=96 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:20:47.969990 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-57648b27-88f4-42ac-aa30-6befc83b47d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbd2c200181b4e40993f6dbfefdd8551', 'request_id': 'req-57648b27-88f4-42ac-aa30-6befc83b47d6'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:47.985096 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-57648b27-88f4-42ac-aa30-6befc83b47d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbd2c200181b4e40993f6dbfefdd8551', 'request_id': 'req-57648b27-88f4-42ac-aa30-6befc83b47d6'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:47.993844 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-57648b27-88f4-42ac-aa30-6befc83b47d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbd2c200181b4e40993f6dbfefdd8551', 'request_id': 'req-57648b27-88f4-42ac-aa30-6befc83b47d6'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:48.008656 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-57648b27-88f4-42ac-aa30-6befc83b47d6 admin admin] The token cache is being invalidate because project fbd2c200181b4e40993f6dbfefdd8551 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 19:20:48.008949 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-57648b27-88f4-42ac-aa30-6befc83b47d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:48.009872 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-57648b27-88f4-42ac-aa30-6befc83b47d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:48.012451 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1101/2198] 10.222.0.61 () {58 vars in 1233 bytes} [Mon Aug 22 19:20:47 2022] DELETE /identity/v3/projects/fbd2c200181b4e40993f6dbfefdd8551 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:20:48.033301 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c819755-183e-4d91-87c3-18381c501f7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:48.033942 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c819755-183e-4d91-87c3-18381c501f7c admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:48.034211 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c819755-183e-4d91-87c3-18381c501f7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:48.034478 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c819755-183e-4d91-87c3-18381c501f7c admin admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:48.035547 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c819755-183e-4d91-87c3-18381c501f7c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:20:48.035911 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c819755-183e-4d91-87c3-18381c501f7c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:48.038561 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c819755-183e-4d91-87c3-18381c501f7c admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:48.054453 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1098/2199] 10.222.0.61 () {58 vars in 1153 bytes} [Mon Aug 22 19:20:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:20:48.062472 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57648b27-88f4-42ac-aa30-6befc83b47d6 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:48.073088 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-57648b27-88f4-42ac-aa30-6befc83b47d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:20:48.090902 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57648b27-88f4-42ac-aa30-6befc83b47d6 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:48.096530 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f9576e-ab5f-440b-aecf-9c6ad238987d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:48.097260 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f9576e-ab5f-440b-aecf-9c6ad238987d admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:48.097603 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f9576e-ab5f-440b-aecf-9c6ad238987d admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:48.097968 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f9576e-ab5f-440b-aecf-9c6ad238987d admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:48.098684 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f9576e-ab5f-440b-aecf-9c6ad238987d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:48.100787 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f9576e-ab5f-440b-aecf-9c6ad238987d admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:48.153596 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1102/2200] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 19:20:48 2022] POST /identity/v3/projects => generated 399 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 19:20:48.157172 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c819755-183e-4d91-87c3-18381c501f7c admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:48.168684 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c819755-183e-4d91-87c3-18381c501f7c admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:48.241132 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70cb09a-e2cc-46fb-b0ca-d954d2e614b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:48.241744 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70cb09a-e2cc-46fb-b0ca-d954d2e614b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:48.242022 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70cb09a-e2cc-46fb-b0ca-d954d2e614b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:48.242294 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70cb09a-e2cc-46fb-b0ca-d954d2e614b8 admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:48.242937 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70cb09a-e2cc-46fb-b0ca-d954d2e614b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:48.244917 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70cb09a-e2cc-46fb-b0ca-d954d2e614b8 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:48.270668 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1099/2201] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 19:20:48 2022] POST /identity/v3/users => generated 300 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:20:48.274763 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f9576e-ab5f-440b-aecf-9c6ad238987d admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:48.285117 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f9576e-ab5f-440b-aecf-9c6ad238987d admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:48.289655 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de2dfc7-7ac9-471b-bef7-78a863980160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:48.290243 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de2dfc7-7ac9-471b-bef7-78a863980160 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:48.290516 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de2dfc7-7ac9-471b-bef7-78a863980160 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:48.290783 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de2dfc7-7ac9-471b-bef7-78a863980160 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:48.291422 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de2dfc7-7ac9-471b-bef7-78a863980160 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:48.293594 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de2dfc7-7ac9-471b-bef7-78a863980160 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:48.304044 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1103/2202] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:20:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:20:48.309383 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70cb09a-e2cc-46fb-b0ca-d954d2e614b8 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:48.319457 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70cb09a-e2cc-46fb-b0ca-d954d2e614b8 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:48.324107 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9aa6dd-6cae-4a74-95b9-0917ea42c381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:48.324705 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9aa6dd-6cae-4a74-95b9-0917ea42c381 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:48.324984 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9aa6dd-6cae-4a74-95b9-0917ea42c381 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:48.325253 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9aa6dd-6cae-4a74-95b9-0917ea42c381 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:48.325890 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9aa6dd-6cae-4a74-95b9-0917ea42c381 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:48.327471 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9aa6dd-6cae-4a74-95b9-0917ea42c381 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:48.338210 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1100/2203] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:20:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:20:48.342258 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de2dfc7-7ac9-471b-bef7-78a863980160 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:48.352455 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de2dfc7-7ac9-471b-bef7-78a863980160 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:48.357594 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0012496-25f0-4d89-afd8-6dfdab244261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:48.358230 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0012496-25f0-4d89-afd8-6dfdab244261 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:48.358503 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0012496-25f0-4d89-afd8-6dfdab244261 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:48.358770 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0012496-25f0-4d89-afd8-6dfdab244261 admin admin] PATH_INFO: `/v3/projects/6e18d67148844528ace04d4c5c3d2416/users/749fc996ff894be9b602a4b367cea975/roles/74a40bc819cb42eea7e966a4b2a016f1` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:48.373852 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0012496-25f0-4d89-afd8-6dfdab244261 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e18d67148844528ace04d4c5c3d2416, user_id=749fc996ff894be9b602a4b367cea975, role_id=74a40bc819cb42eea7e966a4b2a016f1)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:48.376518 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0012496-25f0-4d89-afd8-6dfdab244261 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:48.390120 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1104/2204] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:20:48 2022] PUT /identity/v3/projects/6e18d67148844528ace04d4c5c3d2416/users/749fc996ff894be9b602a4b367cea975/roles/74a40bc819cb42eea7e966a4b2a016f1 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:20:48.401787 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3b8bab-5012-4693-9d38-fc5f177dcbe1 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:48.402112 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3b8bab-5012-4693-9d38-fc5f177dcbe1 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:48.402902 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3b8bab-5012-4693-9d38-fc5f177dcbe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:48.430408 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-9f3b8bab-5012-4693-9d38-fc5f177dcbe1 None None] MFA Rules not processed for user `749fc996ff894be9b602a4b367cea975`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:20:48.431609 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-9f3b8bab-5012-4693-9d38-fc5f177dcbe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:20:48.497659 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1101/2205] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:20:48 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:20:48.508975 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0012496-25f0-4d89-afd8-6dfdab244261 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:48.510698 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-c0012496-25f0-4d89-afd8-6dfdab244261 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:20:48.567757 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0012496-25f0-4d89-afd8-6dfdab244261 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:48.571535 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e2ebf8-6cf7-416d-ac07-d75070e7ec94 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e79498873d4ed29a0e19bbea9b9214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:48.572105 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e2ebf8-6cf7-416d-ac07-d75070e7ec94 service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:48.572380 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e2ebf8-6cf7-416d-ac07-d75070e7ec94 service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:48.572647 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e2ebf8-6cf7-416d-ac07-d75070e7ec94 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:48.581311 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e2ebf8-6cf7-416d-ac07-d75070e7ec94 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:48.582712 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e2ebf8-6cf7-416d-ac07-d75070e7ec94 service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:48.592472 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1105/2206] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:20:48 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:20:54.926654 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3b8bab-5012-4693-9d38-fc5f177dcbe1 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:54.926654 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-9f3b8bab-5012-4693-9d38-fc5f177dcbe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:20:54.940339 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:20:54.992646 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3b8bab-5012-4693-9d38-fc5f177dcbe1 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:54.997658 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee71e01-27fe-4577-bec5-cfc459749d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:54.998279 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee71e01-27fe-4577-bec5-cfc459749d53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:54.998550 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee71e01-27fe-4577-bec5-cfc459749d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:54.998818 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee71e01-27fe-4577-bec5-cfc459749d53 admin admin] PATH_INFO: `/v3/users/4bb5d58c1c2c47d789486609992a77f5` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:55.000032 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee71e01-27fe-4577-bec5-cfc459749d53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4bb5d58c1c2c47d789486609992a77f5)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:55.001927 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee71e01-27fe-4577-bec5-cfc459749d53 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:55.047610 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-0ee71e01-27fe-4577-bec5-cfc459749d53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4bb5d58c1c2c47d789486609992a77f5'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:55.048438 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-0ee71e01-27fe-4577-bec5-cfc459749d53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4bb5d58c1c2c47d789486609992a77f5'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:55.055787 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-0ee71e01-27fe-4577-bec5-cfc459749d53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4bb5d58c1c2c47d789486609992a77f5'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:55.063291 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-0ee71e01-27fe-4577-bec5-cfc459749d53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4bb5d58c1c2c47d789486609992a77f5'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:55.075230 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-0ee71e01-27fe-4577-bec5-cfc459749d53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4bb5d58c1c2c47d789486609992a77f5'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:55.081690 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1102/2207] 10.222.0.61 () {58 vars in 1224 bytes} [Mon Aug 22 19:20:54 2022] DELETE /identity/v3/users/4bb5d58c1c2c47d789486609992a77f5 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:20:55.086139 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ec4281-15c3-4a77-97c5-903e8fd4fa0f None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:55.086612 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ec4281-15c3-4a77-97c5-903e8fd4fa0f None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:55.086741 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ec4281-15c3-4a77-97c5-903e8fd4fa0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:55.106071 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-a2ec4281-15c3-4a77-97c5-903e8fd4fa0f None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:20:55.107269 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-a2ec4281-15c3-4a77-97c5-903e8fd4fa0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:20:55.139306 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:20:55.167451 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1106/2208] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:20:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:20:55.177654 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee71e01-27fe-4577-bec5-cfc459749d53 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:55.179074 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-0ee71e01-27fe-4577-bec5-cfc459749d53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:20:55.235849 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee71e01-27fe-4577-bec5-cfc459749d53 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:55.239582 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11cc38a-37d5-4bab-9fa9-8ec10c486070 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:55.240214 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11cc38a-37d5-4bab-9fa9-8ec10c486070 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:55.240520 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11cc38a-37d5-4bab-9fa9-8ec10c486070 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:55.240792 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11cc38a-37d5-4bab-9fa9-8ec10c486070 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:55.250285 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11cc38a-37d5-4bab-9fa9-8ec10c486070 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:55.251714 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11cc38a-37d5-4bab-9fa9-8ec10c486070 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:55.261143 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1103/2209] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:20:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:20:55.736293 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ec4281-15c3-4a77-97c5-903e8fd4fa0f None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:55.737797 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-a2ec4281-15c3-4a77-97c5-903e8fd4fa0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:20:55.760130 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ec4281-15c3-4a77-97c5-903e8fd4fa0f None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:55.766329 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdad145-bda0-4801-82d2-717d710bd3c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:55.766970 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdad145-bda0-4801-82d2-717d710bd3c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:55.767251 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdad145-bda0-4801-82d2-717d710bd3c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:55.767520 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdad145-bda0-4801-82d2-717d710bd3c2 admin admin] PATH_INFO: `/v3/projects/19936226466c4cb2869ffbc8e2157825` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:55.768672 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdad145-bda0-4801-82d2-717d710bd3c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19936226466c4cb2869ffbc8e2157825)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:55.770556 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdad145-bda0-4801-82d2-717d710bd3c2 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:55.815422 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-efdad145-bda0-4801-82d2-717d710bd3c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19936226466c4cb2869ffbc8e2157825', 'request_id': 'req-efdad145-bda0-4801-82d2-717d710bd3c2'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:55.821612 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-efdad145-bda0-4801-82d2-717d710bd3c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19936226466c4cb2869ffbc8e2157825', 'request_id': 'req-efdad145-bda0-4801-82d2-717d710bd3c2'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:55.828078 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-efdad145-bda0-4801-82d2-717d710bd3c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19936226466c4cb2869ffbc8e2157825', 'request_id': 'req-efdad145-bda0-4801-82d2-717d710bd3c2'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:55.838074 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-efdad145-bda0-4801-82d2-717d710bd3c2 admin admin] The token cache is being invalidate because project 19936226466c4cb2869ffbc8e2157825 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 19:20:55.838356 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-efdad145-bda0-4801-82d2-717d710bd3c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:55.838964 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-efdad145-bda0-4801-82d2-717d710bd3c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:55.840997 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1107/2210] 10.222.0.61 () {58 vars in 1233 bytes} [Mon Aug 22 19:20:55 2022] DELETE /identity/v3/projects/19936226466c4cb2869ffbc8e2157825 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:20:55.845919 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a875972-632f-43e5-a3d2-dc98a97488f3 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:55.846205 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a875972-632f-43e5-a3d2-dc98a97488f3 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:55.846480 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a875972-632f-43e5-a3d2-dc98a97488f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:55.865545 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-6a875972-632f-43e5-a3d2-dc98a97488f3 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:20:55.866741 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-6a875972-632f-43e5-a3d2-dc98a97488f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:20:55.897698 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:20:55.927281 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1104/2211] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:20:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:20:55.935306 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ea4183-b202-4c08-b022-17cf3f3f0585 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:55.935830 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ea4183-b202-4c08-b022-17cf3f3f0585 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:55.936154 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ea4183-b202-4c08-b022-17cf3f3f0585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:55.956361 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-06ea4183-b202-4c08-b022-17cf3f3f0585 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:20:55.957582 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-06ea4183-b202-4c08-b022-17cf3f3f0585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:20:55.968425 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1108/2212] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:20:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:20:55.969703 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a875972-632f-43e5-a3d2-dc98a97488f3 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:55.979932 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a875972-632f-43e5-a3d2-dc98a97488f3 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:55.984398 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5224fe0-db7b-4caf-90b5-e6cd524c0c60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:55.984989 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5224fe0-db7b-4caf-90b5-e6cd524c0c60 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:55.985264 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5224fe0-db7b-4caf-90b5-e6cd524c0c60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:55.985540 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5224fe0-db7b-4caf-90b5-e6cd524c0c60 admin admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:55.986186 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5224fe0-db7b-4caf-90b5-e6cd524c0c60 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:20:55.986526 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5224fe0-db7b-4caf-90b5-e6cd524c0c60 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:55.988153 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5224fe0-db7b-4caf-90b5-e6cd524c0c60 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:55.999941 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1105/2213] 10.222.0.61 () {58 vars in 1153 bytes} [Mon Aug 22 19:20:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:20:56.004093 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ea4183-b202-4c08-b022-17cf3f3f0585 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:56.013891 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ea4183-b202-4c08-b022-17cf3f3f0585 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:56.018557 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1047aae7-122f-4d29-9739-cbfa76d8c4ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:56.019156 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1047aae7-122f-4d29-9739-cbfa76d8c4ec admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:56.019432 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1047aae7-122f-4d29-9739-cbfa76d8c4ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:56.019736 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1047aae7-122f-4d29-9739-cbfa76d8c4ec admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:56.020368 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1047aae7-122f-4d29-9739-cbfa76d8c4ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:56.022246 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1047aae7-122f-4d29-9739-cbfa76d8c4ec admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:56.049129 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1109/2214] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 19:20:56 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 19:20:56.053223 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5224fe0-db7b-4caf-90b5-e6cd524c0c60 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:56.063079 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5224fe0-db7b-4caf-90b5-e6cd524c0c60 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:56.118433 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3f8022-d2d5-43b4-b8ce-95ccb4cfc4d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:56.119040 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3f8022-d2d5-43b4-b8ce-95ccb4cfc4d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:56.119318 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3f8022-d2d5-43b4-b8ce-95ccb4cfc4d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:56.119589 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3f8022-d2d5-43b4-b8ce-95ccb4cfc4d1 admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:56.120241 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3f8022-d2d5-43b4-b8ce-95ccb4cfc4d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:56.122187 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3f8022-d2d5-43b4-b8ce-95ccb4cfc4d1 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:56.142370 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1106/2215] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 19:20:56 2022] POST /identity/v3/users => generated 292 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:20:56.146486 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1047aae7-122f-4d29-9739-cbfa76d8c4ec admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:56.157517 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1047aae7-122f-4d29-9739-cbfa76d8c4ec admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:56.162208 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c98ec3-0377-46c8-bd89-104a705c1e53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:56.162801 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c98ec3-0377-46c8-bd89-104a705c1e53 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:56.163079 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c98ec3-0377-46c8-bd89-104a705c1e53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:56.163346 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c98ec3-0377-46c8-bd89-104a705c1e53 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:56.164016 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c98ec3-0377-46c8-bd89-104a705c1e53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:56.165616 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c98ec3-0377-46c8-bd89-104a705c1e53 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:56.175917 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1110/2216] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:20:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:20:56.179787 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3f8022-d2d5-43b4-b8ce-95ccb4cfc4d1 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:56.189940 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3f8022-d2d5-43b4-b8ce-95ccb4cfc4d1 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:56.194626 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8847eb-40ac-4828-8dcc-a7518c146ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:56.195238 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8847eb-40ac-4828-8dcc-a7518c146ff7 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:56.195514 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8847eb-40ac-4828-8dcc-a7518c146ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:56.195832 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8847eb-40ac-4828-8dcc-a7518c146ff7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:56.196474 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8847eb-40ac-4828-8dcc-a7518c146ff7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:56.198057 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8847eb-40ac-4828-8dcc-a7518c146ff7 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:56.209495 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1107/2217] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:20:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:20:56.213499 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c98ec3-0377-46c8-bd89-104a705c1e53 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:56.224142 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c98ec3-0377-46c8-bd89-104a705c1e53 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:56.228970 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9260cb81-b2be-4b52-88f1-e139a653cb15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:56.229619 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9260cb81-b2be-4b52-88f1-e139a653cb15 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:56.229901 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9260cb81-b2be-4b52-88f1-e139a653cb15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:56.230788 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9260cb81-b2be-4b52-88f1-e139a653cb15 admin admin] PATH_INFO: `/v3/projects/cad7a3404caf4921a38a73acabf9e16e/users/d303ae8ab9f5444db3c0f8434f3cc0fc/roles/74a40bc819cb42eea7e966a4b2a016f1` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:56.244811 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9260cb81-b2be-4b52-88f1-e139a653cb15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cad7a3404caf4921a38a73acabf9e16e, user_id=d303ae8ab9f5444db3c0f8434f3cc0fc, role_id=74a40bc819cb42eea7e966a4b2a016f1)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:56.248015 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9260cb81-b2be-4b52-88f1-e139a653cb15 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:56.260297 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1111/2218] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:20:56 2022] PUT /identity/v3/projects/cad7a3404caf4921a38a73acabf9e16e/users/d303ae8ab9f5444db3c0f8434f3cc0fc/roles/74a40bc819cb42eea7e966a4b2a016f1 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:20:56.270990 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa70f1ee-9144-402c-a064-f9d897e82fdc None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:56.271381 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa70f1ee-9144-402c-a064-f9d897e82fdc None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:56.271678 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa70f1ee-9144-402c-a064-f9d897e82fdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:56.299503 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-fa70f1ee-9144-402c-a064-f9d897e82fdc None None] MFA Rules not processed for user `d303ae8ab9f5444db3c0f8434f3cc0fc`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:20:56.300753 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-fa70f1ee-9144-402c-a064-f9d897e82fdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:20:56.363691 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1108/2219] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:20:56 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:20:56.368148 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9260cb81-b2be-4b52-88f1-e139a653cb15 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:56.424831 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:20:56.428505 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9260cb81-b2be-4b52-88f1-e139a653cb15 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:56.435005 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95568784-d575-41ce-9e6c-1aaf6776133b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:56.435774 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95568784-d575-41ce-9e6c-1aaf6776133b admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:56.436223 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95568784-d575-41ce-9e6c-1aaf6776133b admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:56.436541 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95568784-d575-41ce-9e6c-1aaf6776133b admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:56.437351 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95568784-d575-41ce-9e6c-1aaf6776133b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:56.439415 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95568784-d575-41ce-9e6c-1aaf6776133b admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:56.464748 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1112/2220] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 19:20:56 2022] POST /identity/v3/projects => generated 383 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:20:56.468836 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa70f1ee-9144-402c-a064-f9d897e82fdc None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:56.479965 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa70f1ee-9144-402c-a064-f9d897e82fdc None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:56.509508 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:20:56.538152 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6f0443-1b95-4419-9a29-bd039d284066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:56.538798 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6f0443-1b95-4419-9a29-bd039d284066 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:56.539698 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6f0443-1b95-4419-9a29-bd039d284066 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:56.539698 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6f0443-1b95-4419-9a29-bd039d284066 admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:56.540994 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6f0443-1b95-4419-9a29-bd039d284066 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:56.543306 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6f0443-1b95-4419-9a29-bd039d284066 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:56.567845 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1109/2221] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 19:20:56 2022] POST /identity/v3/users => generated 298 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:20:56.568998 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95568784-d575-41ce-9e6c-1aaf6776133b admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:56.579771 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95568784-d575-41ce-9e6c-1aaf6776133b admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:56.584430 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da75ee9e-2f0e-4875-94de-38dfe944e06c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:56.585027 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da75ee9e-2f0e-4875-94de-38dfe944e06c admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:56.585304 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da75ee9e-2f0e-4875-94de-38dfe944e06c admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:56.585573 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da75ee9e-2f0e-4875-94de-38dfe944e06c admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:56.586209 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da75ee9e-2f0e-4875-94de-38dfe944e06c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:56.587833 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da75ee9e-2f0e-4875-94de-38dfe944e06c admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:56.598336 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1113/2222] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:20:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:20:56.602603 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6f0443-1b95-4419-9a29-bd039d284066 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:56.613011 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6f0443-1b95-4419-9a29-bd039d284066 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:56.617664 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac20d5d-6f2d-4f6e-97af-cdf3de649daa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:56.618304 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac20d5d-6f2d-4f6e-97af-cdf3de649daa admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:56.618582 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac20d5d-6f2d-4f6e-97af-cdf3de649daa admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:56.618853 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac20d5d-6f2d-4f6e-97af-cdf3de649daa admin admin] PATH_INFO: `/v3/projects/c9e48c9ccd0a4b5381d5202be16e7846/users/d3ad061657a449ddb6aeb745e24e7267/roles/163c30c3b3c246beb849ce176f02d9da` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:56.633697 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac20d5d-6f2d-4f6e-97af-cdf3de649daa admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9e48c9ccd0a4b5381d5202be16e7846, user_id=d3ad061657a449ddb6aeb745e24e7267, role_id=163c30c3b3c246beb849ce176f02d9da)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:56.636399 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac20d5d-6f2d-4f6e-97af-cdf3de649daa admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:56.648659 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1110/2223] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:20:56 2022] PUT /identity/v3/projects/c9e48c9ccd0a4b5381d5202be16e7846/users/d3ad061657a449ddb6aeb745e24e7267/roles/163c30c3b3c246beb849ce176f02d9da => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:20:56.652960 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da75ee9e-2f0e-4875-94de-38dfe944e06c admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:56.662845 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da75ee9e-2f0e-4875-94de-38dfe944e06c admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:56.719524 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ece19b-93f9-49e7-8b06-1e714a12bdb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:56.720162 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ece19b-93f9-49e7-8b06-1e714a12bdb0 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:56.720441 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ece19b-93f9-49e7-8b06-1e714a12bdb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:56.720711 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ece19b-93f9-49e7-8b06-1e714a12bdb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:56.721350 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ece19b-93f9-49e7-8b06-1e714a12bdb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:56.722938 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ece19b-93f9-49e7-8b06-1e714a12bdb0 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:56.736724 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1114/2224] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:20:56 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:20:56.737876 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac20d5d-6f2d-4f6e-97af-cdf3de649daa admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:56.748560 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac20d5d-6f2d-4f6e-97af-cdf3de649daa admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:56.753172 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269e9e21-b550-481e-85d1-ca4b6e7e542a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:56.753824 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269e9e21-b550-481e-85d1-ca4b6e7e542a admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:56.754099 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269e9e21-b550-481e-85d1-ca4b6e7e542a admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:56.754370 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269e9e21-b550-481e-85d1-ca4b6e7e542a admin admin] PATH_INFO: `/v3/projects/c9e48c9ccd0a4b5381d5202be16e7846/users/d3ad061657a449ddb6aeb745e24e7267/roles/163c30c3b3c246beb849ce176f02d9da` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:56.756292 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269e9e21-b550-481e-85d1-ca4b6e7e542a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9e48c9ccd0a4b5381d5202be16e7846, user_id=d3ad061657a449ddb6aeb745e24e7267, role_id=163c30c3b3c246beb849ce176f02d9da)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:56.758913 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269e9e21-b550-481e-85d1-ca4b6e7e542a admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:56.770001 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1111/2225] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:20:56 2022] PUT /identity/v3/projects/c9e48c9ccd0a4b5381d5202be16e7846/users/d3ad061657a449ddb6aeb745e24e7267/roles/163c30c3b3c246beb849ce176f02d9da => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:20:56.781662 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae09738b-813e-4650-bc14-d40dfb907307 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:56.781953 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae09738b-813e-4650-bc14-d40dfb907307 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:56.782229 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae09738b-813e-4650-bc14-d40dfb907307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:56.813189 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-ae09738b-813e-4650-bc14-d40dfb907307 None None] MFA Rules not processed for user `d3ad061657a449ddb6aeb745e24e7267`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:20:56.814440 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-ae09738b-813e-4650-bc14-d40dfb907307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:20:56.887204 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1115/2226] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:20:56 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:20:56.898541 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269e9e21-b550-481e-85d1-ca4b6e7e542a admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:56.900260 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-269e9e21-b550-481e-85d1-ca4b6e7e542a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:20:56.958890 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269e9e21-b550-481e-85d1-ca4b6e7e542a admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:56.962615 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24ceb6f-73e2-48d9-83ff-dd59cd473fc8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e79498873d4ed29a0e19bbea9b9214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:56.963209 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24ceb6f-73e2-48d9-83ff-dd59cd473fc8 service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:56.963486 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24ceb6f-73e2-48d9-83ff-dd59cd473fc8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:56.963789 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24ceb6f-73e2-48d9-83ff-dd59cd473fc8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:56.973052 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24ceb6f-73e2-48d9-83ff-dd59cd473fc8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:56.974457 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24ceb6f-73e2-48d9-83ff-dd59cd473fc8 service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:56.984876 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1112/2227] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:20:56 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:20:57.054661 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae09738b-813e-4650-bc14-d40dfb907307 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:57.056215 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-ae09738b-813e-4650-bc14-d40dfb907307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:20:57.068197 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae09738b-813e-4650-bc14-d40dfb907307 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:57.071931 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91623bc-400f-4896-8bd6-0e896595e131 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e79498873d4ed29a0e19bbea9b9214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:57.072522 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91623bc-400f-4896-8bd6-0e896595e131 service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:57.072804 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91623bc-400f-4896-8bd6-0e896595e131 service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:57.073075 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91623bc-400f-4896-8bd6-0e896595e131 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:57.121346 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:20:57.131816 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91623bc-400f-4896-8bd6-0e896595e131 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:57.133305 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91623bc-400f-4896-8bd6-0e896595e131 service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:57.142804 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1116/2228] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:20:57 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:20:57.399241 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24ceb6f-73e2-48d9-83ff-dd59cd473fc8 service cinder] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:57.460782 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24ceb6f-73e2-48d9-83ff-dd59cd473fc8 service cinder] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:57.465778 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1371077a-1100-4781-83d7-8cdff0c67f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:57.466404 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1371077a-1100-4781-83d7-8cdff0c67f62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:57.466678 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1371077a-1100-4781-83d7-8cdff0c67f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:57.467956 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1371077a-1100-4781-83d7-8cdff0c67f62 admin admin] PATH_INFO: `/v3/users/d303ae8ab9f5444db3c0f8434f3cc0fc` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:57.469135 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1371077a-1100-4781-83d7-8cdff0c67f62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d303ae8ab9f5444db3c0f8434f3cc0fc)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:57.471082 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1371077a-1100-4781-83d7-8cdff0c67f62 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:57.500312 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:20:57.519353 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-1371077a-1100-4781-83d7-8cdff0c67f62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd303ae8ab9f5444db3c0f8434f3cc0fc'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:57.520018 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-1371077a-1100-4781-83d7-8cdff0c67f62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd303ae8ab9f5444db3c0f8434f3cc0fc'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:57.526896 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-1371077a-1100-4781-83d7-8cdff0c67f62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd303ae8ab9f5444db3c0f8434f3cc0fc'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:57.534629 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-1371077a-1100-4781-83d7-8cdff0c67f62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd303ae8ab9f5444db3c0f8434f3cc0fc'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:57.547434 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-1371077a-1100-4781-83d7-8cdff0c67f62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd303ae8ab9f5444db3c0f8434f3cc0fc'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:57.553391 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1113/2229] 10.222.0.61 () {58 vars in 1224 bytes} [Mon Aug 22 19:20:57 2022] DELETE /identity/v3/users/d303ae8ab9f5444db3c0f8434f3cc0fc => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:20:57.557898 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e0fecd-2526-49c0-aa34-8fc017185907 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:57.558361 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e0fecd-2526-49c0-aa34-8fc017185907 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:57.558453 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e0fecd-2526-49c0-aa34-8fc017185907 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:57.577685 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-66e0fecd-2526-49c0-aa34-8fc017185907 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:20:57.579374 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-66e0fecd-2526-49c0-aa34-8fc017185907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:20:57.640240 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1117/2230] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:20:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:20:57.650273 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1371077a-1100-4781-83d7-8cdff0c67f62 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:57.651755 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-1371077a-1100-4781-83d7-8cdff0c67f62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:20:57.708410 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1371077a-1100-4781-83d7-8cdff0c67f62 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:57.712086 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b6bae8-a738-4196-846f-e476e7c79f5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:57.712683 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b6bae8-a738-4196-846f-e476e7c79f5e service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:57.712961 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b6bae8-a738-4196-846f-e476e7c79f5e service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:57.713232 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b6bae8-a738-4196-846f-e476e7c79f5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:57.722833 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b6bae8-a738-4196-846f-e476e7c79f5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:57.724302 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b6bae8-a738-4196-846f-e476e7c79f5e service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:57.734773 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1114/2231] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:20:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:20:58.068823 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e0fecd-2526-49c0-aa34-8fc017185907 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:58.070593 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-66e0fecd-2526-49c0-aa34-8fc017185907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:20:58.087201 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e0fecd-2526-49c0-aa34-8fc017185907 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:58.092489 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e5b875-fd79-407b-9011-fcaeef096716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:58.093127 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e5b875-fd79-407b-9011-fcaeef096716 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:58.093398 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e5b875-fd79-407b-9011-fcaeef096716 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:58.093665 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e5b875-fd79-407b-9011-fcaeef096716 admin admin] PATH_INFO: `/v3/projects/cad7a3404caf4921a38a73acabf9e16e` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:58.094802 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e5b875-fd79-407b-9011-fcaeef096716 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cad7a3404caf4921a38a73acabf9e16e)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:58.096761 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e5b875-fd79-407b-9011-fcaeef096716 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:58.154616 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-18e5b875-fd79-407b-9011-fcaeef096716 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cad7a3404caf4921a38a73acabf9e16e', 'request_id': 'req-18e5b875-fd79-407b-9011-fcaeef096716'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:58.162059 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-18e5b875-fd79-407b-9011-fcaeef096716 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cad7a3404caf4921a38a73acabf9e16e', 'request_id': 'req-18e5b875-fd79-407b-9011-fcaeef096716'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:58.168909 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-18e5b875-fd79-407b-9011-fcaeef096716 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cad7a3404caf4921a38a73acabf9e16e', 'request_id': 'req-18e5b875-fd79-407b-9011-fcaeef096716'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:58.179070 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-18e5b875-fd79-407b-9011-fcaeef096716 admin admin] The token cache is being invalidate because project cad7a3404caf4921a38a73acabf9e16e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 19:20:58.179353 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-18e5b875-fd79-407b-9011-fcaeef096716 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:58.180007 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-18e5b875-fd79-407b-9011-fcaeef096716 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:58.181633 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1118/2232] 10.222.0.61 () {58 vars in 1233 bytes} [Mon Aug 22 19:20:58 2022] DELETE /identity/v3/projects/cad7a3404caf4921a38a73acabf9e16e => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:20:58.185703 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b6bae8-a738-4196-846f-e476e7c79f5e service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:58.195487 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b6bae8-a738-4196-846f-e476e7c79f5e service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:58.251139 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1775063d-8e0a-452a-9096-eb5af9627b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:58.251797 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1775063d-8e0a-452a-9096-eb5af9627b9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:58.252077 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1775063d-8e0a-452a-9096-eb5af9627b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:58.252346 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1775063d-8e0a-452a-9096-eb5af9627b9a admin admin] PATH_INFO: `/v3/users/d3ad061657a449ddb6aeb745e24e7267` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:58.253447 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1775063d-8e0a-452a-9096-eb5af9627b9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3ad061657a449ddb6aeb745e24e7267)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:58.255343 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1775063d-8e0a-452a-9096-eb5af9627b9a admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:58.301530 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-1775063d-8e0a-452a-9096-eb5af9627b9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3ad061657a449ddb6aeb745e24e7267'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:58.302217 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-1775063d-8e0a-452a-9096-eb5af9627b9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3ad061657a449ddb6aeb745e24e7267'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:58.304295 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:20:58.310229 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-1775063d-8e0a-452a-9096-eb5af9627b9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3ad061657a449ddb6aeb745e24e7267'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:58.317361 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-1775063d-8e0a-452a-9096-eb5af9627b9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3ad061657a449ddb6aeb745e24e7267'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:58.329228 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-1775063d-8e0a-452a-9096-eb5af9627b9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3ad061657a449ddb6aeb745e24e7267'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:58.335054 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1115/2233] 10.222.0.61 () {58 vars in 1224 bytes} [Mon Aug 22 19:20:58 2022] DELETE /identity/v3/users/d3ad061657a449ddb6aeb745e24e7267 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:20:58.603206 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e5b875-fd79-407b-9011-fcaeef096716 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:58.604710 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-18e5b875-fd79-407b-9011-fcaeef096716 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:20:58.619475 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e5b875-fd79-407b-9011-fcaeef096716 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:58.679486 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5988f0-b9ce-40f5-878b-9d3f186e49b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:58.680162 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5988f0-b9ce-40f5-878b-9d3f186e49b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:58.680460 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5988f0-b9ce-40f5-878b-9d3f186e49b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:58.680721 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5988f0-b9ce-40f5-878b-9d3f186e49b1 admin admin] PATH_INFO: `/v3/projects/c9e48c9ccd0a4b5381d5202be16e7846` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:58.681844 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5988f0-b9ce-40f5-878b-9d3f186e49b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9e48c9ccd0a4b5381d5202be16e7846)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:58.684383 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5988f0-b9ce-40f5-878b-9d3f186e49b1 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:58.700807 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:20:58.731348 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-ca5988f0-b9ce-40f5-878b-9d3f186e49b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9e48c9ccd0a4b5381d5202be16e7846', 'request_id': 'req-ca5988f0-b9ce-40f5-878b-9d3f186e49b1'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:58.738805 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-ca5988f0-b9ce-40f5-878b-9d3f186e49b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9e48c9ccd0a4b5381d5202be16e7846', 'request_id': 'req-ca5988f0-b9ce-40f5-878b-9d3f186e49b1'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:58.745193 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-ca5988f0-b9ce-40f5-878b-9d3f186e49b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9e48c9ccd0a4b5381d5202be16e7846', 'request_id': 'req-ca5988f0-b9ce-40f5-878b-9d3f186e49b1'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:58.756040 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-ca5988f0-b9ce-40f5-878b-9d3f186e49b1 admin admin] The token cache is being invalidate because project c9e48c9ccd0a4b5381d5202be16e7846 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 19:20:58.756331 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-ca5988f0-b9ce-40f5-878b-9d3f186e49b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:58.756955 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-ca5988f0-b9ce-40f5-878b-9d3f186e49b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:20:58.758433 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1119/2234] 10.222.0.61 () {58 vars in 1233 bytes} [Mon Aug 22 19:20:58 2022] DELETE /identity/v3/projects/c9e48c9ccd0a4b5381d5202be16e7846 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:20:58.763252 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61a5b56-5033-4cef-9df6-35bc29ef0b30 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:58.763539 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61a5b56-5033-4cef-9df6-35bc29ef0b30 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:58.763855 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61a5b56-5033-4cef-9df6-35bc29ef0b30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:58.783945 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-e61a5b56-5033-4cef-9df6-35bc29ef0b30 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:20:58.785124 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-e61a5b56-5033-4cef-9df6-35bc29ef0b30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:20:58.842807 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1116/2235] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:20:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:20:58.850543 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6ed487-f145-48ba-b0c4-a9ab93c98b9e None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:58.851023 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6ed487-f145-48ba-b0c4-a9ab93c98b9e None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:58.851309 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6ed487-f145-48ba-b0c4-a9ab93c98b9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:58.871449 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-6d6ed487-f145-48ba-b0c4-a9ab93c98b9e None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:20:58.872692 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-6d6ed487-f145-48ba-b0c4-a9ab93c98b9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:20:58.880586 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1120/2236] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:20:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:20:58.884611 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61a5b56-5033-4cef-9df6-35bc29ef0b30 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:58.894745 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61a5b56-5033-4cef-9df6-35bc29ef0b30 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:58.899701 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f7a35d-67ef-4f21-9bc1-72bac4e56187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:58.900301 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f7a35d-67ef-4f21-9bc1-72bac4e56187 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:58.900580 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f7a35d-67ef-4f21-9bc1-72bac4e56187 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:58.900851 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f7a35d-67ef-4f21-9bc1-72bac4e56187 admin admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:58.901505 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f7a35d-67ef-4f21-9bc1-72bac4e56187 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:20:58.901813 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f7a35d-67ef-4f21-9bc1-72bac4e56187 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:58.904001 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f7a35d-67ef-4f21-9bc1-72bac4e56187 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:58.915701 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1117/2237] 10.222.0.61 () {58 vars in 1153 bytes} [Mon Aug 22 19:20:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 19:20:58.920565 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6ed487-f145-48ba-b0c4-a9ab93c98b9e None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:58.930124 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6ed487-f145-48ba-b0c4-a9ab93c98b9e None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:58.934883 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb80eec-10d4-4ccf-a2e4-a41d032a37ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:58.935482 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb80eec-10d4-4ccf-a2e4-a41d032a37ef admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:58.935806 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb80eec-10d4-4ccf-a2e4-a41d032a37ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:58.936081 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb80eec-10d4-4ccf-a2e4-a41d032a37ef admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:58.936738 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb80eec-10d4-4ccf-a2e4-a41d032a37ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:58.938733 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb80eec-10d4-4ccf-a2e4-a41d032a37ef admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:58.965857 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1121/2238] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 19:20:58 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:20:58.970181 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f7a35d-67ef-4f21-9bc1-72bac4e56187 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:58.981007 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f7a35d-67ef-4f21-9bc1-72bac4e56187 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:59.036835 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cac114-a356-446d-aa86-5fbaf1f6318a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:59.037435 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cac114-a356-446d-aa86-5fbaf1f6318a admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:59.037715 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cac114-a356-446d-aa86-5fbaf1f6318a admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:59.037990 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cac114-a356-446d-aa86-5fbaf1f6318a admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:59.038616 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cac114-a356-446d-aa86-5fbaf1f6318a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:59.040563 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cac114-a356-446d-aa86-5fbaf1f6318a admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:59.062023 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1118/2239] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 19:20:58 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 19:20:59.066265 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb80eec-10d4-4ccf-a2e4-a41d032a37ef admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:59.076446 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb80eec-10d4-4ccf-a2e4-a41d032a37ef admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:59.081021 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71d11e5-c020-44e2-bd78-93b908a1c8d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:59.081794 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71d11e5-c020-44e2-bd78-93b908a1c8d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:59.082147 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71d11e5-c020-44e2-bd78-93b908a1c8d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:59.082446 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71d11e5-c020-44e2-bd78-93b908a1c8d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:59.083109 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71d11e5-c020-44e2-bd78-93b908a1c8d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:59.084731 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71d11e5-c020-44e2-bd78-93b908a1c8d3 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:59.095830 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1122/2240] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:20:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:20:59.099917 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cac114-a356-446d-aa86-5fbaf1f6318a admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:59.110911 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cac114-a356-446d-aa86-5fbaf1f6318a admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:59.116132 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5173f9-4ffd-4a08-a9c6-cd0155ea1abd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:59.116741 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5173f9-4ffd-4a08-a9c6-cd0155ea1abd admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:59.117020 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5173f9-4ffd-4a08-a9c6-cd0155ea1abd admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:59.117321 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5173f9-4ffd-4a08-a9c6-cd0155ea1abd admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:59.117971 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5173f9-4ffd-4a08-a9c6-cd0155ea1abd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:59.119555 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5173f9-4ffd-4a08-a9c6-cd0155ea1abd admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:59.131216 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1119/2241] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:20:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:20:59.142650 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71d11e5-c020-44e2-bd78-93b908a1c8d3 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:59.159047 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71d11e5-c020-44e2-bd78-93b908a1c8d3 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:59.164695 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d8a1ac-bc3c-47a3-8d51-fa0120f5e890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:59.166857 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d8a1ac-bc3c-47a3-8d51-fa0120f5e890 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:59.167237 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d8a1ac-bc3c-47a3-8d51-fa0120f5e890 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:59.168148 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d8a1ac-bc3c-47a3-8d51-fa0120f5e890 admin admin] PATH_INFO: `/v3/projects/575a2f7426a44c0da3de70245d63076f/users/96a6b33a16c749d7ac7f63d06afe0d31/roles/74a40bc819cb42eea7e966a4b2a016f1` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:59.184311 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d8a1ac-bc3c-47a3-8d51-fa0120f5e890 admin admin] RBAC: Authorizing `identity:create_grant(project_id=575a2f7426a44c0da3de70245d63076f, user_id=96a6b33a16c749d7ac7f63d06afe0d31, role_id=74a40bc819cb42eea7e966a4b2a016f1)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:59.187164 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d8a1ac-bc3c-47a3-8d51-fa0120f5e890 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:59.200795 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1123/2242] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:20:59 2022] PUT /identity/v3/projects/575a2f7426a44c0da3de70245d63076f/users/96a6b33a16c749d7ac7f63d06afe0d31/roles/74a40bc819cb42eea7e966a4b2a016f1 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:20:59.211118 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0b57ee-2939-4349-b63f-6ac0a86002aa None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:59.211689 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0b57ee-2939-4349-b63f-6ac0a86002aa None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:59.211689 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0b57ee-2939-4349-b63f-6ac0a86002aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:59.240609 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-9b0b57ee-2939-4349-b63f-6ac0a86002aa None None] MFA Rules not processed for user `96a6b33a16c749d7ac7f63d06afe0d31`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:20:59.241843 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-9b0b57ee-2939-4349-b63f-6ac0a86002aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:20:59.262906 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:20:59.309014 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1120/2243] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:20:59 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:20:59.313550 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d8a1ac-bc3c-47a3-8d51-fa0120f5e890 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:59.376900 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d8a1ac-bc3c-47a3-8d51-fa0120f5e890 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:59.381748 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0e5053-3c3f-42cd-b796-4e8613ca524f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:59.382392 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0e5053-3c3f-42cd-b796-4e8613ca524f admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:59.382672 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0e5053-3c3f-42cd-b796-4e8613ca524f admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:59.383000 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0e5053-3c3f-42cd-b796-4e8613ca524f admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:59.383628 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0e5053-3c3f-42cd-b796-4e8613ca524f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:59.385656 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0e5053-3c3f-42cd-b796-4e8613ca524f admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:59.413172 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1124/2244] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 19:20:59 2022] POST /identity/v3/projects => generated 385 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:20:59.417828 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0b57ee-2939-4349-b63f-6ac0a86002aa None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:59.429014 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0b57ee-2939-4349-b63f-6ac0a86002aa None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:59.488733 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784c9b28-5890-4e20-92a3-8270aa3b14c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:59.489391 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784c9b28-5890-4e20-92a3-8270aa3b14c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:59.489695 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784c9b28-5890-4e20-92a3-8270aa3b14c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:59.490236 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784c9b28-5890-4e20-92a3-8270aa3b14c1 admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:59.490865 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784c9b28-5890-4e20-92a3-8270aa3b14c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:59.492841 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784c9b28-5890-4e20-92a3-8270aa3b14c1 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:59.514757 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1121/2245] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 19:20:59 2022] POST /identity/v3/users => generated 299 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 19:20:59.518746 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0e5053-3c3f-42cd-b796-4e8613ca524f admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:59.530185 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0e5053-3c3f-42cd-b796-4e8613ca524f admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:59.535319 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de2f2c9-e347-4145-9759-496b0fc18f47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:59.535949 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de2f2c9-e347-4145-9759-496b0fc18f47 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:59.536223 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de2f2c9-e347-4145-9759-496b0fc18f47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:59.536490 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de2f2c9-e347-4145-9759-496b0fc18f47 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:59.537128 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de2f2c9-e347-4145-9759-496b0fc18f47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:59.538708 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de2f2c9-e347-4145-9759-496b0fc18f47 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:59.550296 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1125/2246] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:20:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:20:59.554392 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784c9b28-5890-4e20-92a3-8270aa3b14c1 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:59.567510 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784c9b28-5890-4e20-92a3-8270aa3b14c1 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:59.572943 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33653598-53b7-44e0-8c6f-ad89f309e56c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:59.573636 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33653598-53b7-44e0-8c6f-ad89f309e56c admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:59.574441 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33653598-53b7-44e0-8c6f-ad89f309e56c admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:59.574709 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33653598-53b7-44e0-8c6f-ad89f309e56c admin admin] PATH_INFO: `/v3/projects/5d2aa732961d4d9790b8543368c290b9/users/091b24e6fcb64353894157772a32d092/roles/163c30c3b3c246beb849ce176f02d9da` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:59.592090 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33653598-53b7-44e0-8c6f-ad89f309e56c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d2aa732961d4d9790b8543368c290b9, user_id=091b24e6fcb64353894157772a32d092, role_id=163c30c3b3c246beb849ce176f02d9da)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:59.594714 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33653598-53b7-44e0-8c6f-ad89f309e56c admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:59.607268 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1122/2247] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:20:59 2022] PUT /identity/v3/projects/5d2aa732961d4d9790b8543368c290b9/users/091b24e6fcb64353894157772a32d092/roles/163c30c3b3c246beb849ce176f02d9da => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 19:20:59.611246 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de2f2c9-e347-4145-9759-496b0fc18f47 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:59.621916 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de2f2c9-e347-4145-9759-496b0fc18f47 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:59.633511 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:20:59.681544 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bab0ef2-c370-4024-a9bf-b8f1eb946a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:59.682136 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bab0ef2-c370-4024-a9bf-b8f1eb946a09 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:59.682414 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bab0ef2-c370-4024-a9bf-b8f1eb946a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:59.682681 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bab0ef2-c370-4024-a9bf-b8f1eb946a09 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:59.683871 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bab0ef2-c370-4024-a9bf-b8f1eb946a09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:59.685499 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bab0ef2-c370-4024-a9bf-b8f1eb946a09 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:59.695716 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1126/2248] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:20:59 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:20:59.700351 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33653598-53b7-44e0-8c6f-ad89f309e56c admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:59.711379 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33653598-53b7-44e0-8c6f-ad89f309e56c admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:59.716228 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b45b7b4-8e31-4be7-9f76-4df074e3dd8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:59.716875 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b45b7b4-8e31-4be7-9f76-4df074e3dd8f admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:59.717148 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b45b7b4-8e31-4be7-9f76-4df074e3dd8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:59.717417 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b45b7b4-8e31-4be7-9f76-4df074e3dd8f admin admin] PATH_INFO: `/v3/projects/5d2aa732961d4d9790b8543368c290b9/users/091b24e6fcb64353894157772a32d092/roles/163c30c3b3c246beb849ce176f02d9da` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:59.719382 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b45b7b4-8e31-4be7-9f76-4df074e3dd8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d2aa732961d4d9790b8543368c290b9, user_id=091b24e6fcb64353894157772a32d092, role_id=163c30c3b3c246beb849ce176f02d9da)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:59.722425 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b45b7b4-8e31-4be7-9f76-4df074e3dd8f admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:59.733990 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1123/2249] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:20:59 2022] PUT /identity/v3/projects/5d2aa732961d4d9790b8543368c290b9/users/091b24e6fcb64353894157772a32d092/roles/163c30c3b3c246beb849ce176f02d9da => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:20:59.746224 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c23083-bc2c-4bf8-b0ce-7f99a42068e3 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:59.746638 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c23083-bc2c-4bf8-b0ce-7f99a42068e3 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:59.747008 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c23083-bc2c-4bf8-b0ce-7f99a42068e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:59.775325 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-08c23083-bc2c-4bf8-b0ce-7f99a42068e3 None None] MFA Rules not processed for user `091b24e6fcb64353894157772a32d092`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:20:59.776594 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-08c23083-bc2c-4bf8-b0ce-7f99a42068e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:20:59.844997 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1127/2250] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:20:59 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:20:59.856451 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b45b7b4-8e31-4be7-9f76-4df074e3dd8f admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:20:59.858148 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-4b45b7b4-8e31-4be7-9f76-4df074e3dd8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:20:59.914545 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b45b7b4-8e31-4be7-9f76-4df074e3dd8f admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:20:59.918845 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be58b2d5-81ef-4c72-8a15-aaf5095cfb81 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e79498873d4ed29a0e19bbea9b9214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:20:59.919445 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be58b2d5-81ef-4c72-8a15-aaf5095cfb81 service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:20:59.919760 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be58b2d5-81ef-4c72-8a15-aaf5095cfb81 service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:20:59.920035 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be58b2d5-81ef-4c72-8a15-aaf5095cfb81 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:20:59.948411 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:20:59.983665 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be58b2d5-81ef-4c72-8a15-aaf5095cfb81 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:20:59.985188 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be58b2d5-81ef-4c72-8a15-aaf5095cfb81 service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:20:59.995312 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1124/2251] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:20:59 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:21:00.346741 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c23083-bc2c-4bf8-b0ce-7f99a42068e3 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:21:00.348597 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-08c23083-bc2c-4bf8-b0ce-7f99a42068e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:21:00.480938 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c23083-bc2c-4bf8-b0ce-7f99a42068e3 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:21:00.488860 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d39a2e8-db13-4daf-8784-af06a56fee00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:21:00.489605 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d39a2e8-db13-4daf-8784-af06a56fee00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:21:00.489978 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d39a2e8-db13-4daf-8784-af06a56fee00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:21:00.490344 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d39a2e8-db13-4daf-8784-af06a56fee00 admin admin] PATH_INFO: `/v3/users/8682aca1fc4f4b8a9e244e84b3676d94` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:21:00.491603 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d39a2e8-db13-4daf-8784-af06a56fee00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8682aca1fc4f4b8a9e244e84b3676d94)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:21:00.493638 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d39a2e8-db13-4daf-8784-af06a56fee00 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:21:00.544259 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:21:00.551446 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-0d39a2e8-db13-4daf-8784-af06a56fee00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8682aca1fc4f4b8a9e244e84b3676d94'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:21:00.552159 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-0d39a2e8-db13-4daf-8784-af06a56fee00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8682aca1fc4f4b8a9e244e84b3676d94'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:21:00.559039 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-0d39a2e8-db13-4daf-8784-af06a56fee00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8682aca1fc4f4b8a9e244e84b3676d94'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:21:00.566139 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-0d39a2e8-db13-4daf-8784-af06a56fee00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8682aca1fc4f4b8a9e244e84b3676d94'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:21:00.579065 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-0d39a2e8-db13-4daf-8784-af06a56fee00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8682aca1fc4f4b8a9e244e84b3676d94'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:21:00.587507 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1128/2252] 10.222.0.61 () {58 vars in 1224 bytes} [Mon Aug 22 19:21:00 2022] DELETE /identity/v3/users/8682aca1fc4f4b8a9e244e84b3676d94 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:21:00.592454 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9462abaf-6478-4c13-83f0-4d5e1a834e61 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:21:00.592988 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9462abaf-6478-4c13-83f0-4d5e1a834e61 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:21:00.592988 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9462abaf-6478-4c13-83f0-4d5e1a834e61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:21:00.613399 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-9462abaf-6478-4c13-83f0-4d5e1a834e61 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:21:00.614627 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-9462abaf-6478-4c13-83f0-4d5e1a834e61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:21:00.675356 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1125/2253] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:21:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:21:00.684928 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d39a2e8-db13-4daf-8784-af06a56fee00 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:21:00.686464 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-0d39a2e8-db13-4daf-8784-af06a56fee00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:21:00.744924 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d39a2e8-db13-4daf-8784-af06a56fee00 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:21:00.748614 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9af814e-067b-44aa-b44d-d8518febb7c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:21:00.749216 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9af814e-067b-44aa-b44d-d8518febb7c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:21:00.749491 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9af814e-067b-44aa-b44d-d8518febb7c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:21:00.749765 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9af814e-067b-44aa-b44d-d8518febb7c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:21:00.758621 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9af814e-067b-44aa-b44d-d8518febb7c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:21:00.760072 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9af814e-067b-44aa-b44d-d8518febb7c1 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:21:00.769855 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1129/2254] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:21:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:21:01.055605 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9462abaf-6478-4c13-83f0-4d5e1a834e61 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:21:01.057428 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-9462abaf-6478-4c13-83f0-4d5e1a834e61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:21:01.073536 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9462abaf-6478-4c13-83f0-4d5e1a834e61 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:21:01.078609 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6672263-5ff0-4b6d-a29d-1f0e94599a5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:21:01.079282 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6672263-5ff0-4b6d-a29d-1f0e94599a5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:21:01.079569 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6672263-5ff0-4b6d-a29d-1f0e94599a5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:21:01.079888 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6672263-5ff0-4b6d-a29d-1f0e94599a5d admin admin] PATH_INFO: `/v3/projects/1815855da8ee442bb4674d155de57b7f` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:21:01.081119 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6672263-5ff0-4b6d-a29d-1f0e94599a5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=1815855da8ee442bb4674d155de57b7f)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:21:01.083218 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6672263-5ff0-4b6d-a29d-1f0e94599a5d admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:21:01.137660 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-c6672263-5ff0-4b6d-a29d-1f0e94599a5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1815855da8ee442bb4674d155de57b7f', 'request_id': 'req-c6672263-5ff0-4b6d-a29d-1f0e94599a5d'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:21:01.148001 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-c6672263-5ff0-4b6d-a29d-1f0e94599a5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1815855da8ee442bb4674d155de57b7f', 'request_id': 'req-c6672263-5ff0-4b6d-a29d-1f0e94599a5d'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:21:01.154954 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-c6672263-5ff0-4b6d-a29d-1f0e94599a5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1815855da8ee442bb4674d155de57b7f', 'request_id': 'req-c6672263-5ff0-4b6d-a29d-1f0e94599a5d'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:21:01.165145 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-c6672263-5ff0-4b6d-a29d-1f0e94599a5d admin admin] The token cache is being invalidate because project 1815855da8ee442bb4674d155de57b7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 19:21:01.165431 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-c6672263-5ff0-4b6d-a29d-1f0e94599a5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:21:01.166060 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-c6672263-5ff0-4b6d-a29d-1f0e94599a5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:21:01.167580 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1126/2255] 10.222.0.61 () {58 vars in 1233 bytes} [Mon Aug 22 19:21:01 2022] DELETE /identity/v3/projects/1815855da8ee442bb4674d155de57b7f => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:21:01.173285 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e21d4d-eac4-434f-93de-ae102ed34402 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:21:01.173570 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e21d4d-eac4-434f-93de-ae102ed34402 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:21:01.173868 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e21d4d-eac4-434f-93de-ae102ed34402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:21:01.193321 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-96e21d4d-eac4-434f-93de-ae102ed34402 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:21:01.194520 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-96e21d4d-eac4-434f-93de-ae102ed34402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:21:01.253967 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1130/2256] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:21:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:21:01.261800 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f726ce-87b6-4444-8972-4da5a31d5635 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:21:01.262084 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f726ce-87b6-4444-8972-4da5a31d5635 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:21:01.262355 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f726ce-87b6-4444-8972-4da5a31d5635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:21:01.282206 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-a7f726ce-87b6-4444-8972-4da5a31d5635 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:21:01.283420 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-a7f726ce-87b6-4444-8972-4da5a31d5635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:21:01.291451 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1127/2257] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:21:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:21:01.296312 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e21d4d-eac4-434f-93de-ae102ed34402 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:21:01.306021 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e21d4d-eac4-434f-93de-ae102ed34402 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:21:01.310566 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5532cd2-9870-4a1d-b357-458f0fac285c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:21:01.311160 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5532cd2-9870-4a1d-b357-458f0fac285c admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:21:01.311440 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5532cd2-9870-4a1d-b357-458f0fac285c admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:21:01.311745 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5532cd2-9870-4a1d-b357-458f0fac285c admin admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:21:01.312398 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5532cd2-9870-4a1d-b357-458f0fac285c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:21:01.312700 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5532cd2-9870-4a1d-b357-458f0fac285c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:21:01.314294 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5532cd2-9870-4a1d-b357-458f0fac285c admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:21:01.325930 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1131/2258] 10.222.0.61 () {58 vars in 1153 bytes} [Mon Aug 22 19:21:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:21:01.329965 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f726ce-87b6-4444-8972-4da5a31d5635 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:21:01.340056 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f726ce-87b6-4444-8972-4da5a31d5635 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:21:01.344881 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b493bc-ab43-42c4-b8a0-607a845ed683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:21:01.345529 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b493bc-ab43-42c4-b8a0-607a845ed683 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:21:01.345767 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b493bc-ab43-42c4-b8a0-607a845ed683 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:21:01.346060 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b493bc-ab43-42c4-b8a0-607a845ed683 admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:21:01.346697 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b493bc-ab43-42c4-b8a0-607a845ed683 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:21:01.348730 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b493bc-ab43-42c4-b8a0-607a845ed683 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:21:01.375411 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1128/2259] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 19:21:01 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:21:01.380099 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5532cd2-9870-4a1d-b357-458f0fac285c admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:21:01.390914 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5532cd2-9870-4a1d-b357-458f0fac285c admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:21:01.438153 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:21:01.454516 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6c87cf-5f59-4eff-86e8-f00f9d3aa02d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:21:01.455155 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6c87cf-5f59-4eff-86e8-f00f9d3aa02d admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:21:01.455373 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6c87cf-5f59-4eff-86e8-f00f9d3aa02d admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:21:01.455658 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6c87cf-5f59-4eff-86e8-f00f9d3aa02d admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:21:01.456295 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6c87cf-5f59-4eff-86e8-f00f9d3aa02d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:21:01.458204 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6c87cf-5f59-4eff-86e8-f00f9d3aa02d admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:21:01.494138 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1132/2260] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 19:21:01 2022] POST /identity/v3/users => generated 288 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:21:01.495716 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b493bc-ab43-42c4-b8a0-607a845ed683 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:21:01.507364 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b493bc-ab43-42c4-b8a0-607a845ed683 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:21:01.514015 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37461085-7b4e-4cb4-847f-0e6791a28c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:21:01.514631 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37461085-7b4e-4cb4-847f-0e6791a28c55 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:21:01.514914 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37461085-7b4e-4cb4-847f-0e6791a28c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:21:01.515180 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37461085-7b4e-4cb4-847f-0e6791a28c55 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:21:01.516047 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37461085-7b4e-4cb4-847f-0e6791a28c55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:21:01.517656 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37461085-7b4e-4cb4-847f-0e6791a28c55 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:21:01.529627 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1129/2261] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:21:01 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:21:01.534048 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6c87cf-5f59-4eff-86e8-f00f9d3aa02d admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:21:01.546592 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6c87cf-5f59-4eff-86e8-f00f9d3aa02d admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:21:01.552665 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094e3123-5c63-495f-a0c8-6a9f0fae6d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:21:01.553276 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e3123-5c63-495f-a0c8-6a9f0fae6d28 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:21:01.553553 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e3123-5c63-495f-a0c8-6a9f0fae6d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:21:01.553828 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e3123-5c63-495f-a0c8-6a9f0fae6d28 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:21:01.554476 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094e3123-5c63-495f-a0c8-6a9f0fae6d28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:21:01.556127 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094e3123-5c63-495f-a0c8-6a9f0fae6d28 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:21:01.569831 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1133/2262] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:21:01 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:21:01.574277 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37461085-7b4e-4cb4-847f-0e6791a28c55 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:21:01.585757 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37461085-7b4e-4cb4-847f-0e6791a28c55 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:21:01.592308 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c173a4d-7c76-4fff-9807-f12925c49dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:21:01.593143 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c173a4d-7c76-4fff-9807-f12925c49dc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:21:01.593874 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c173a4d-7c76-4fff-9807-f12925c49dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:21:01.596035 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c173a4d-7c76-4fff-9807-f12925c49dc0 admin admin] PATH_INFO: `/v3/projects/495fd878c0ee42b38f082ccdcb9ca384/users/d5e2e66799164a90a5efc14624a85c7c/roles/74a40bc819cb42eea7e966a4b2a016f1` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:21:01.609180 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:21:01.614455 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c173a4d-7c76-4fff-9807-f12925c49dc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=495fd878c0ee42b38f082ccdcb9ca384, user_id=d5e2e66799164a90a5efc14624a85c7c, role_id=74a40bc819cb42eea7e966a4b2a016f1)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:21:01.617590 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c173a4d-7c76-4fff-9807-f12925c49dc0 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:21:01.639424 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1130/2263] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:21:01 2022] PUT /identity/v3/projects/495fd878c0ee42b38f082ccdcb9ca384/users/d5e2e66799164a90a5efc14624a85c7c/roles/74a40bc819cb42eea7e966a4b2a016f1 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:21:01.652673 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9624f4-a4bf-4a8f-80cd-8c3e588eddf6 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:21:01.652987 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9624f4-a4bf-4a8f-80cd-8c3e588eddf6 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:21:01.653259 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9624f4-a4bf-4a8f-80cd-8c3e588eddf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:21:01.687109 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-7f9624f4-a4bf-4a8f-80cd-8c3e588eddf6 None None] MFA Rules not processed for user `d5e2e66799164a90a5efc14624a85c7c`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:21:01.688651 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-7f9624f4-a4bf-4a8f-80cd-8c3e588eddf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:21:01.778131 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1134/2264] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:21:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:21:01.789389 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c173a4d-7c76-4fff-9807-f12925c49dc0 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:21:01.791175 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-3c173a4d-7c76-4fff-9807-f12925c49dc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:21:01.883486 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c173a4d-7c76-4fff-9807-f12925c49dc0 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:21:01.888528 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce3b58c-f4f4-412a-98a7-7150936ae724 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e79498873d4ed29a0e19bbea9b9214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:21:01.889186 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce3b58c-f4f4-412a-98a7-7150936ae724 service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:21:01.889516 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce3b58c-f4f4-412a-98a7-7150936ae724 service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:21:01.889708 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce3b58c-f4f4-412a-98a7-7150936ae724 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:21:01.898991 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce3b58c-f4f4-412a-98a7-7150936ae724 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:21:01.900440 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce3b58c-f4f4-412a-98a7-7150936ae724 service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:21:01.913052 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1131/2265] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:21:01 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:21:02.186200 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9624f4-a4bf-4a8f-80cd-8c3e588eddf6 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:21:02.188649 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-7f9624f4-a4bf-4a8f-80cd-8c3e588eddf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:21:02.285591 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9624f4-a4bf-4a8f-80cd-8c3e588eddf6 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:21:02.293480 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0208636d-6935-4f74-b5be-e4d8eaf2ad67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:21:02.294226 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0208636d-6935-4f74-b5be-e4d8eaf2ad67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:21:02.294502 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0208636d-6935-4f74-b5be-e4d8eaf2ad67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:21:02.294771 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0208636d-6935-4f74-b5be-e4d8eaf2ad67 admin admin] PATH_INFO: `/v3/users/749fc996ff894be9b602a4b367cea975` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:21:02.296066 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0208636d-6935-4f74-b5be-e4d8eaf2ad67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=749fc996ff894be9b602a4b367cea975)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:21:02.297970 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0208636d-6935-4f74-b5be-e4d8eaf2ad67 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:21:02.353573 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-0208636d-6935-4f74-b5be-e4d8eaf2ad67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '749fc996ff894be9b602a4b367cea975'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:21:02.354320 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-0208636d-6935-4f74-b5be-e4d8eaf2ad67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '749fc996ff894be9b602a4b367cea975'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:21:02.363360 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-0208636d-6935-4f74-b5be-e4d8eaf2ad67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '749fc996ff894be9b602a4b367cea975'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:21:02.365228 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:21:02.372932 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-0208636d-6935-4f74-b5be-e4d8eaf2ad67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '749fc996ff894be9b602a4b367cea975'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:21:02.387822 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-0208636d-6935-4f74-b5be-e4d8eaf2ad67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '749fc996ff894be9b602a4b367cea975'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:21:02.395506 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1135/2266] 10.222.0.61 () {58 vars in 1224 bytes} [Mon Aug 22 19:21:02 2022] DELETE /identity/v3/users/749fc996ff894be9b602a4b367cea975 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 19:21:02.400347 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae74dcd-2422-4636-83f5-fa0bf4d5cb37 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:21:02.400739 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae74dcd-2422-4636-83f5-fa0bf4d5cb37 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:21:02.401106 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae74dcd-2422-4636-83f5-fa0bf4d5cb37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:21:02.420277 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-cae74dcd-2422-4636-83f5-fa0bf4d5cb37 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:21:02.421481 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-cae74dcd-2422-4636-83f5-fa0bf4d5cb37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:21:02.480508 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1132/2267] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:21:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:21:02.490004 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0208636d-6935-4f74-b5be-e4d8eaf2ad67 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:21:02.491415 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-0208636d-6935-4f74-b5be-e4d8eaf2ad67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:21:02.549603 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0208636d-6935-4f74-b5be-e4d8eaf2ad67 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:21:02.553403 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e422c050-8a8c-422f-8e0f-b939937f1cbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:21:02.554012 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e422c050-8a8c-422f-8e0f-b939937f1cbd service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:21:02.554291 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e422c050-8a8c-422f-8e0f-b939937f1cbd service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:21:02.554561 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e422c050-8a8c-422f-8e0f-b939937f1cbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:21:02.564599 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e422c050-8a8c-422f-8e0f-b939937f1cbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:21:02.566093 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e422c050-8a8c-422f-8e0f-b939937f1cbd service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:21:02.575931 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1136/2268] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:21:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:21:02.848206 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae74dcd-2422-4636-83f5-fa0bf4d5cb37 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:21:02.849852 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-cae74dcd-2422-4636-83f5-fa0bf4d5cb37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:21:02.863228 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae74dcd-2422-4636-83f5-fa0bf4d5cb37 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:21:02.867913 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e9ac4b-0f44-4272-aeef-3a8c170cc485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:21:02.868558 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e9ac4b-0f44-4272-aeef-3a8c170cc485 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:21:02.868812 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e9ac4b-0f44-4272-aeef-3a8c170cc485 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:21:02.869125 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e9ac4b-0f44-4272-aeef-3a8c170cc485 admin admin] PATH_INFO: `/v3/projects/6e18d67148844528ace04d4c5c3d2416` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:21:02.870449 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e9ac4b-0f44-4272-aeef-3a8c170cc485 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e18d67148844528ace04d4c5c3d2416)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:21:02.872807 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e9ac4b-0f44-4272-aeef-3a8c170cc485 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:21:02.922890 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-89e9ac4b-0f44-4272-aeef-3a8c170cc485 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e18d67148844528ace04d4c5c3d2416', 'request_id': 'req-89e9ac4b-0f44-4272-aeef-3a8c170cc485'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:21:02.931532 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-89e9ac4b-0f44-4272-aeef-3a8c170cc485 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e18d67148844528ace04d4c5c3d2416', 'request_id': 'req-89e9ac4b-0f44-4272-aeef-3a8c170cc485'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:21:02.938891 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-89e9ac4b-0f44-4272-aeef-3a8c170cc485 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e18d67148844528ace04d4c5c3d2416', 'request_id': 'req-89e9ac4b-0f44-4272-aeef-3a8c170cc485'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:21:02.949520 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-89e9ac4b-0f44-4272-aeef-3a8c170cc485 admin admin] The token cache is being invalidate because project 6e18d67148844528ace04d4c5c3d2416 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 19:21:02.949807 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-89e9ac4b-0f44-4272-aeef-3a8c170cc485 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:21:02.950483 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-89e9ac4b-0f44-4272-aeef-3a8c170cc485 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:21:02.951997 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1133/2269] 10.222.0.61 () {58 vars in 1233 bytes} [Mon Aug 22 19:21:02 2022] DELETE /identity/v3/projects/6e18d67148844528ace04d4c5c3d2416 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:21:02.957102 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddd714e-0f00-4883-836d-759b4d9859ac None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:21:02.957558 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddd714e-0f00-4883-836d-759b4d9859ac None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:21:02.957937 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddd714e-0f00-4883-836d-759b4d9859ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:21:02.979754 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-9ddd714e-0f00-4883-836d-759b4d9859ac None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:21:02.981583 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-9ddd714e-0f00-4883-836d-759b4d9859ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:21:03.056063 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1137/2270] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:21:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:21:03.063936 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171fa770-491a-4169-951c-8077df0ff66b None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:21:03.064315 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171fa770-491a-4169-951c-8077df0ff66b None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:21:03.064657 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171fa770-491a-4169-951c-8077df0ff66b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:21:03.085678 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-171fa770-491a-4169-951c-8077df0ff66b None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:21:03.087016 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-171fa770-491a-4169-951c-8077df0ff66b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:21:03.096094 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1134/2271] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:21:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:21:03.100410 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddd714e-0f00-4883-836d-759b4d9859ac None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:21:03.111275 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddd714e-0f00-4883-836d-759b4d9859ac None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:21:03.116482 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719a97ff-977d-4370-a6be-902c96c3c017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:21:03.117107 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719a97ff-977d-4370-a6be-902c96c3c017 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:21:03.117390 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719a97ff-977d-4370-a6be-902c96c3c017 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:21:03.117698 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719a97ff-977d-4370-a6be-902c96c3c017 admin admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:21:03.118329 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719a97ff-977d-4370-a6be-902c96c3c017 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:21:03.118634 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719a97ff-977d-4370-a6be-902c96c3c017 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:21:03.120292 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719a97ff-977d-4370-a6be-902c96c3c017 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:21:03.133884 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1138/2272] 10.222.0.61 () {58 vars in 1153 bytes} [Mon Aug 22 19:21:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:21:03.140336 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171fa770-491a-4169-951c-8077df0ff66b None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:21:03.154140 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171fa770-491a-4169-951c-8077df0ff66b None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:21:03.159551 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c563860f-cdbe-478c-915d-a2d1ade0185f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:21:03.160258 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c563860f-cdbe-478c-915d-a2d1ade0185f admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:21:03.160535 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c563860f-cdbe-478c-915d-a2d1ade0185f admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:21:03.160806 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c563860f-cdbe-478c-915d-a2d1ade0185f admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:21:03.161449 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c563860f-cdbe-478c-915d-a2d1ade0185f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:21:03.163635 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c563860f-cdbe-478c-915d-a2d1ade0185f admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:21:03.170636 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:21:03.202466 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1135/2273] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 19:21:03 2022] POST /identity/v3/projects => generated 377 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:21:03.203967 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719a97ff-977d-4370-a6be-902c96c3c017 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:21:03.214442 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719a97ff-977d-4370-a6be-902c96c3c017 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:21:03.302861 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f0125e-c2a9-4eaf-b86d-32bed97d87da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:21:03.303527 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f0125e-c2a9-4eaf-b86d-32bed97d87da admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:21:03.303870 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f0125e-c2a9-4eaf-b86d-32bed97d87da admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:21:03.304073 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f0125e-c2a9-4eaf-b86d-32bed97d87da admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:21:03.304698 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f0125e-c2a9-4eaf-b86d-32bed97d87da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:21:03.306606 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f0125e-c2a9-4eaf-b86d-32bed97d87da admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:21:03.322243 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:INSERT=3,keystone:DELETE=3,keystone:SELECT=94 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:21:03.342749 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1139/2274] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 19:21:03 2022] POST /identity/v3/users => generated 289 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:21:03.346544 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c563860f-cdbe-478c-915d-a2d1ade0185f admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:21:03.358697 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c563860f-cdbe-478c-915d-a2d1ade0185f admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:21:03.363290 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b55dbd-33bb-4210-a3fb-f0f464fe95d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:21:03.363936 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b55dbd-33bb-4210-a3fb-f0f464fe95d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:21:03.364219 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b55dbd-33bb-4210-a3fb-f0f464fe95d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:21:03.364549 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b55dbd-33bb-4210-a3fb-f0f464fe95d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:21:03.365194 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b55dbd-33bb-4210-a3fb-f0f464fe95d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:21:03.366865 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b55dbd-33bb-4210-a3fb-f0f464fe95d9 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:21:03.377762 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1136/2275] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:21:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 19:21:03.381931 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f0125e-c2a9-4eaf-b86d-32bed97d87da admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:21:03.392662 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f0125e-c2a9-4eaf-b86d-32bed97d87da admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:21:03.398660 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76a1223-1ec3-423c-bee0-b8f422c35e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:21:03.399392 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76a1223-1ec3-423c-bee0-b8f422c35e34 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:21:03.399783 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76a1223-1ec3-423c-bee0-b8f422c35e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:21:03.399951 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76a1223-1ec3-423c-bee0-b8f422c35e34 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:21:03.400726 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76a1223-1ec3-423c-bee0-b8f422c35e34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:21:03.402881 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76a1223-1ec3-423c-bee0-b8f422c35e34 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:21:03.413715 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1140/2276] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:21:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:21:03.418406 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b55dbd-33bb-4210-a3fb-f0f464fe95d9 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:21:03.428355 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b55dbd-33bb-4210-a3fb-f0f464fe95d9 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:21:03.434317 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0af0c5a-2b83-48af-9040-906398b475f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:21:03.435038 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0af0c5a-2b83-48af-9040-906398b475f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:21:03.435317 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0af0c5a-2b83-48af-9040-906398b475f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:21:03.435590 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0af0c5a-2b83-48af-9040-906398b475f5 admin admin] PATH_INFO: `/v3/projects/e3adbe4feef048239c2020ac458e11b0/users/a1b3705aa22743c49e3ac2f37c7f2084/roles/74a40bc819cb42eea7e966a4b2a016f1` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:21:03.450793 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0af0c5a-2b83-48af-9040-906398b475f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3adbe4feef048239c2020ac458e11b0, user_id=a1b3705aa22743c49e3ac2f37c7f2084, role_id=74a40bc819cb42eea7e966a4b2a016f1)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:21:03.453481 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0af0c5a-2b83-48af-9040-906398b475f5 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:21:03.466617 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1137/2277] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:21:03 2022] PUT /identity/v3/projects/e3adbe4feef048239c2020ac458e11b0/users/a1b3705aa22743c49e3ac2f37c7f2084/roles/74a40bc819cb42eea7e966a4b2a016f1 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:21:03.471546 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c3b691-3a09-407f-8d60-27a0b917fcaf None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:21:03.471854 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c3b691-3a09-407f-8d60-27a0b917fcaf None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:21:03.472128 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c3b691-3a09-407f-8d60-27a0b917fcaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:21:03.490539 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-78c3b691-3a09-407f-8d60-27a0b917fcaf None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:21:03.491758 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-78c3b691-3a09-407f-8d60-27a0b917fcaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:21:03.549120 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1141/2278] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:21:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:21:03.559706 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0af0c5a-2b83-48af-9040-906398b475f5 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:21:03.561807 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-b0af0c5a-2b83-48af-9040-906398b475f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:21:03.623969 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0af0c5a-2b83-48af-9040-906398b475f5 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:21:03.630821 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54daecce-5eb3-4a21-95f2-df5b1cef5111 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:21:03.631441 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54daecce-5eb3-4a21-95f2-df5b1cef5111 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:21:03.631771 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54daecce-5eb3-4a21-95f2-df5b1cef5111 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:21:03.632046 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54daecce-5eb3-4a21-95f2-df5b1cef5111 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:21:03.642373 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54daecce-5eb3-4a21-95f2-df5b1cef5111 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:21:03.643861 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54daecce-5eb3-4a21-95f2-df5b1cef5111 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:21:03.653170 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1138/2279] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:21:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:21:05.513519 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c3b691-3a09-407f-8d60-27a0b917fcaf None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:21:05.515189 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-78c3b691-3a09-407f-8d60-27a0b917fcaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:21:05.580628 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c3b691-3a09-407f-8d60-27a0b917fcaf None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:21:05.585163 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abf666d-c1c4-458d-8c2f-0c1a0ede9feb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e79498873d4ed29a0e19bbea9b9214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:21:05.585785 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abf666d-c1c4-458d-8c2f-0c1a0ede9feb service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:21:05.586087 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abf666d-c1c4-458d-8c2f-0c1a0ede9feb service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:21:05.586363 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abf666d-c1c4-458d-8c2f-0c1a0ede9feb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:21:05.588147 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-2abf666d-c1c4-458d-8c2f-0c1a0ede9feb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:21:05.680874 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abf666d-c1c4-458d-8c2f-0c1a0ede9feb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:21:05.682747 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abf666d-c1c4-458d-8c2f-0c1a0ede9feb service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:21:05.708039 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1142/2280] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:21:05 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:21:08.238729 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cf0664-3115-4f84-9c6a-c928c01ff588 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:21:08.239236 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cf0664-3115-4f84-9c6a-c928c01ff588 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:21:08.239619 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cf0664-3115-4f84-9c6a-c928c01ff588 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:21:08.276382 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-42cf0664-3115-4f84-9c6a-c928c01ff588 None None] MFA Rules not processed for user `a1b3705aa22743c49e3ac2f37c7f2084`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:21:08.278321 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-42cf0664-3115-4f84-9c6a-c928c01ff588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:21:08.356655 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1139/2281] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:21:08 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:21:08.361219 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abf666d-c1c4-458d-8c2f-0c1a0ede9feb service cinder] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:21:08.372475 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abf666d-c1c4-458d-8c2f-0c1a0ede9feb service cinder] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:21:08.380399 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbf4b84-4846-438c-8e1d-81b9547e2f50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:21:08.381030 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbf4b84-4846-438c-8e1d-81b9547e2f50 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:21:08.381308 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbf4b84-4846-438c-8e1d-81b9547e2f50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:21:08.381782 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbf4b84-4846-438c-8e1d-81b9547e2f50 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:21:08.382421 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbf4b84-4846-438c-8e1d-81b9547e2f50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:21:08.384386 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbf4b84-4846-438c-8e1d-81b9547e2f50 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:21:08.416853 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1143/2282] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 19:21:08 2022] POST /identity/v3/projects => generated 379 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:21:08.421428 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cf0664-3115-4f84-9c6a-c928c01ff588 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:21:08.434269 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cf0664-3115-4f84-9c6a-c928c01ff588 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:21:08.488994 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:21:08.507162 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8be7196-35fd-4d24-a99c-fbfd9d2bfa89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:21:08.507822 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8be7196-35fd-4d24-a99c-fbfd9d2bfa89 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:21:08.508105 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8be7196-35fd-4d24-a99c-fbfd9d2bfa89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:21:08.508379 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8be7196-35fd-4d24-a99c-fbfd9d2bfa89 admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:21:08.509019 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8be7196-35fd-4d24-a99c-fbfd9d2bfa89 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:21:08.511048 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8be7196-35fd-4d24-a99c-fbfd9d2bfa89 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:21:08.532867 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1140/2283] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 19:21:08 2022] POST /identity/v3/users => generated 296 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:21:08.537026 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbf4b84-4846-438c-8e1d-81b9547e2f50 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:21:08.548442 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbf4b84-4846-438c-8e1d-81b9547e2f50 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:21:08.553062 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa8aab4-db8d-4815-820e-c9f98dc40a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:21:08.553681 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa8aab4-db8d-4815-820e-c9f98dc40a0f admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:21:08.553965 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa8aab4-db8d-4815-820e-c9f98dc40a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:21:08.554231 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa8aab4-db8d-4815-820e-c9f98dc40a0f admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:21:08.554884 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa8aab4-db8d-4815-820e-c9f98dc40a0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:21:08.556527 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa8aab4-db8d-4815-820e-c9f98dc40a0f admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:21:08.562989 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:21:08.569921 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1144/2284] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:21:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:21:08.575334 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8be7196-35fd-4d24-a99c-fbfd9d2bfa89 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:21:08.587310 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8be7196-35fd-4d24-a99c-fbfd9d2bfa89 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:21:08.593190 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663b797d-8116-466b-9471-9f517dcca457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:21:08.593942 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663b797d-8116-466b-9471-9f517dcca457 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:21:08.594286 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663b797d-8116-466b-9471-9f517dcca457 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:21:08.594629 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663b797d-8116-466b-9471-9f517dcca457 admin admin] PATH_INFO: `/v3/projects/97b0943c57664ba6acbdc01928fc178e/users/d4deb36470284a368a6bf4fde96c6a33/roles/163c30c3b3c246beb849ce176f02d9da` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:21:08.611778 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663b797d-8116-466b-9471-9f517dcca457 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97b0943c57664ba6acbdc01928fc178e, user_id=d4deb36470284a368a6bf4fde96c6a33, role_id=163c30c3b3c246beb849ce176f02d9da)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:21:08.614434 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663b797d-8116-466b-9471-9f517dcca457 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:21:08.629241 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1141/2285] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:21:08 2022] PUT /identity/v3/projects/97b0943c57664ba6acbdc01928fc178e/users/d4deb36470284a368a6bf4fde96c6a33/roles/163c30c3b3c246beb849ce176f02d9da => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:21:08.633353 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa8aab4-db8d-4815-820e-c9f98dc40a0f admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:21:08.644379 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa8aab4-db8d-4815-820e-c9f98dc40a0f admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:21:08.719161 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e50fc8c-b383-43e6-9244-092122bfde51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:21:08.719829 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e50fc8c-b383-43e6-9244-092122bfde51 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:21:08.720113 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e50fc8c-b383-43e6-9244-092122bfde51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:21:08.720386 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e50fc8c-b383-43e6-9244-092122bfde51 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:21:08.721072 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e50fc8c-b383-43e6-9244-092122bfde51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:21:08.722677 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e50fc8c-b383-43e6-9244-092122bfde51 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:21:08.737161 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1145/2286] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:21:08 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:21:08.741065 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663b797d-8116-466b-9471-9f517dcca457 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:21:08.762207 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663b797d-8116-466b-9471-9f517dcca457 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:21:08.767011 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6750f461-29a0-4e8a-9fd2-7b16cc450d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:21:08.767727 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6750f461-29a0-4e8a-9fd2-7b16cc450d16 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:21:08.768006 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6750f461-29a0-4e8a-9fd2-7b16cc450d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:21:08.768282 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6750f461-29a0-4e8a-9fd2-7b16cc450d16 admin admin] PATH_INFO: `/v3/projects/97b0943c57664ba6acbdc01928fc178e/users/d4deb36470284a368a6bf4fde96c6a33/roles/163c30c3b3c246beb849ce176f02d9da` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:21:08.770384 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6750f461-29a0-4e8a-9fd2-7b16cc450d16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97b0943c57664ba6acbdc01928fc178e, user_id=d4deb36470284a368a6bf4fde96c6a33, role_id=163c30c3b3c246beb849ce176f02d9da)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:21:08.773067 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6750f461-29a0-4e8a-9fd2-7b16cc450d16 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:21:08.784500 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1142/2287] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:21:08 2022] PUT /identity/v3/projects/97b0943c57664ba6acbdc01928fc178e/users/d4deb36470284a368a6bf4fde96c6a33/roles/163c30c3b3c246beb849ce176f02d9da => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:21:13.545615 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27532414-8473-4b05-9a4f-e3d6850fba8e None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:21:13.546163 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27532414-8473-4b05-9a4f-e3d6850fba8e None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:21:13.546163 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27532414-8473-4b05-9a4f-e3d6850fba8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:21:13.573892 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-27532414-8473-4b05-9a4f-e3d6850fba8e None None] MFA Rules not processed for user `d4deb36470284a368a6bf4fde96c6a33`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:21:13.575153 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-27532414-8473-4b05-9a4f-e3d6850fba8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:21:13.653391 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1146/2288] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:21:13 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:21:13.675060 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6750f461-29a0-4e8a-9fd2-7b16cc450d16 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:21:13.677340 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-6750f461-29a0-4e8a-9fd2-7b16cc450d16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:21:13.781950 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6750f461-29a0-4e8a-9fd2-7b16cc450d16 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:21:13.788802 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2665eb3c-3807-4995-a651-317a6e81491f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f43e2756b746a6825ca1a06b7d7c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:21:13.789795 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2665eb3c-3807-4995-a651-317a6e81491f service nova] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:21:13.789795 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2665eb3c-3807-4995-a651-317a6e81491f service nova] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:21:13.789795 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2665eb3c-3807-4995-a651-317a6e81491f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:21:13.871973 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2665eb3c-3807-4995-a651-317a6e81491f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:21:13.874006 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2665eb3c-3807-4995-a651-317a6e81491f service nova] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:21:13.895053 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1143/2289] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:21:13 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:21:21.497854 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27532414-8473-4b05-9a4f-e3d6850fba8e None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:21:21.499296 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-27532414-8473-4b05-9a4f-e3d6850fba8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:21:21.566433 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27532414-8473-4b05-9a4f-e3d6850fba8e None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:21:21.572425 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3903713-ac98-4bcc-a80c-90d6090d3c48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:21:21.573182 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3903713-ac98-4bcc-a80c-90d6090d3c48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:21:21.573469 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3903713-ac98-4bcc-a80c-90d6090d3c48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:21:21.573755 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3903713-ac98-4bcc-a80c-90d6090d3c48 admin admin] PATH_INFO: `/v3/users/96a6b33a16c749d7ac7f63d06afe0d31` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:21:21.575475 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3903713-ac98-4bcc-a80c-90d6090d3c48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=96a6b33a16c749d7ac7f63d06afe0d31)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:21:21.577547 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3903713-ac98-4bcc-a80c-90d6090d3c48 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:21:21.601934 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:21:21.629884 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-d3903713-ac98-4bcc-a80c-90d6090d3c48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96a6b33a16c749d7ac7f63d06afe0d31'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:21:21.630562 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-d3903713-ac98-4bcc-a80c-90d6090d3c48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96a6b33a16c749d7ac7f63d06afe0d31'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:21:21.638412 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-d3903713-ac98-4bcc-a80c-90d6090d3c48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96a6b33a16c749d7ac7f63d06afe0d31'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:21:21.648785 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-d3903713-ac98-4bcc-a80c-90d6090d3c48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96a6b33a16c749d7ac7f63d06afe0d31'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:21:21.661378 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-d3903713-ac98-4bcc-a80c-90d6090d3c48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96a6b33a16c749d7ac7f63d06afe0d31'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:21:21.668029 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1147/2290] 10.222.0.61 () {58 vars in 1224 bytes} [Mon Aug 22 19:21:21 2022] DELETE /identity/v3/users/96a6b33a16c749d7ac7f63d06afe0d31 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:21:21.675770 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f03d779-7ed4-413c-82a3-d68687433a84 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:21:21.676338 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f03d779-7ed4-413c-82a3-d68687433a84 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:21:21.676338 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f03d779-7ed4-413c-82a3-d68687433a84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:21:21.695470 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-2f03d779-7ed4-413c-82a3-d68687433a84 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:21:21.696708 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-2f03d779-7ed4-413c-82a3-d68687433a84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:21:21.737207 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:21:21.773632 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1144/2291] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:21:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:21:21.783344 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3903713-ac98-4bcc-a80c-90d6090d3c48 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:21:21.784872 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-d3903713-ac98-4bcc-a80c-90d6090d3c48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:21:21.845470 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3903713-ac98-4bcc-a80c-90d6090d3c48 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:21:21.849128 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cbf577-6323-4f64-86b3-84d5840b16ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:21:21.849737 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cbf577-6323-4f64-86b3-84d5840b16ed service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:21:21.850016 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cbf577-6323-4f64-86b3-84d5840b16ed service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:21:21.850287 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cbf577-6323-4f64-86b3-84d5840b16ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:21:21.860638 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cbf577-6323-4f64-86b3-84d5840b16ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:21:21.862195 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cbf577-6323-4f64-86b3-84d5840b16ed service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:21:21.873538 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1148/2292] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:21:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:21:22.162425 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f03d779-7ed4-413c-82a3-d68687433a84 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:21:22.164388 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-2f03d779-7ed4-413c-82a3-d68687433a84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:21:22.179005 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f03d779-7ed4-413c-82a3-d68687433a84 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:21:22.186894 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775089eb-b345-4faf-8d3e-485bbc6ddb85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:21:22.187747 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775089eb-b345-4faf-8d3e-485bbc6ddb85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:21:22.188223 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775089eb-b345-4faf-8d3e-485bbc6ddb85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:21:22.188600 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775089eb-b345-4faf-8d3e-485bbc6ddb85 admin admin] PATH_INFO: `/v3/projects/575a2f7426a44c0da3de70245d63076f` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:21:22.189908 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775089eb-b345-4faf-8d3e-485bbc6ddb85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=575a2f7426a44c0da3de70245d63076f)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:21:22.192282 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775089eb-b345-4faf-8d3e-485bbc6ddb85 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:21:22.258707 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-775089eb-b345-4faf-8d3e-485bbc6ddb85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '575a2f7426a44c0da3de70245d63076f', 'request_id': 'req-775089eb-b345-4faf-8d3e-485bbc6ddb85'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:21:22.268807 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-775089eb-b345-4faf-8d3e-485bbc6ddb85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '575a2f7426a44c0da3de70245d63076f', 'request_id': 'req-775089eb-b345-4faf-8d3e-485bbc6ddb85'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:21:22.277773 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-775089eb-b345-4faf-8d3e-485bbc6ddb85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '575a2f7426a44c0da3de70245d63076f', 'request_id': 'req-775089eb-b345-4faf-8d3e-485bbc6ddb85'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:21:22.299442 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-775089eb-b345-4faf-8d3e-485bbc6ddb85 admin admin] The token cache is being invalidate because project 575a2f7426a44c0da3de70245d63076f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 19:21:22.299774 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-775089eb-b345-4faf-8d3e-485bbc6ddb85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:21:22.300450 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-775089eb-b345-4faf-8d3e-485bbc6ddb85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:21:22.311225 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1145/2293] 10.222.0.61 () {58 vars in 1233 bytes} [Mon Aug 22 19:21:22 2022] DELETE /identity/v3/projects/575a2f7426a44c0da3de70245d63076f => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:21:22.312373 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cbf577-6323-4f64-86b3-84d5840b16ed service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:21:22.326265 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cbf577-6323-4f64-86b3-84d5840b16ed service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:21:22.392077 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd78390-9d0b-4277-ae29-31942ef89936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:21:22.392709 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd78390-9d0b-4277-ae29-31942ef89936 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:21:22.392982 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd78390-9d0b-4277-ae29-31942ef89936 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:21:22.393257 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd78390-9d0b-4277-ae29-31942ef89936 admin admin] PATH_INFO: `/v3/users/091b24e6fcb64353894157772a32d092` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:21:22.394401 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd78390-9d0b-4277-ae29-31942ef89936 admin admin] RBAC: Authorizing `identity:delete_user(user_id=091b24e6fcb64353894157772a32d092)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:21:22.396363 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd78390-9d0b-4277-ae29-31942ef89936 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:21:22.442747 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:21:22.444484 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-6bd78390-9d0b-4277-ae29-31942ef89936 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '091b24e6fcb64353894157772a32d092'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:21:22.445372 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-6bd78390-9d0b-4277-ae29-31942ef89936 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '091b24e6fcb64353894157772a32d092'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:21:22.453594 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-6bd78390-9d0b-4277-ae29-31942ef89936 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '091b24e6fcb64353894157772a32d092'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:21:22.461264 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-6bd78390-9d0b-4277-ae29-31942ef89936 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '091b24e6fcb64353894157772a32d092'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:21:22.474197 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-6bd78390-9d0b-4277-ae29-31942ef89936 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '091b24e6fcb64353894157772a32d092'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:21:22.481988 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1149/2294] 10.222.0.61 () {58 vars in 1224 bytes} [Mon Aug 22 19:21:22 2022] DELETE /identity/v3/users/091b24e6fcb64353894157772a32d092 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:21:22.772386 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775089eb-b345-4faf-8d3e-485bbc6ddb85 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:21:22.773849 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-775089eb-b345-4faf-8d3e-485bbc6ddb85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:21:22.788536 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775089eb-b345-4faf-8d3e-485bbc6ddb85 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:21:22.847734 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c82b6f7-f547-4a72-8c52-8d7462f0a63d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:21:22.848375 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c82b6f7-f547-4a72-8c52-8d7462f0a63d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:21:22.848665 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c82b6f7-f547-4a72-8c52-8d7462f0a63d admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:21:22.848936 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c82b6f7-f547-4a72-8c52-8d7462f0a63d admin admin] PATH_INFO: `/v3/projects/5d2aa732961d4d9790b8543368c290b9` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:21:22.850178 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c82b6f7-f547-4a72-8c52-8d7462f0a63d admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d2aa732961d4d9790b8543368c290b9)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:21:22.852392 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c82b6f7-f547-4a72-8c52-8d7462f0a63d admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:21:22.900136 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-5c82b6f7-f547-4a72-8c52-8d7462f0a63d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d2aa732961d4d9790b8543368c290b9', 'request_id': 'req-5c82b6f7-f547-4a72-8c52-8d7462f0a63d'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:21:22.907914 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-5c82b6f7-f547-4a72-8c52-8d7462f0a63d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d2aa732961d4d9790b8543368c290b9', 'request_id': 'req-5c82b6f7-f547-4a72-8c52-8d7462f0a63d'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:21:22.910209 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:21:22.914536 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-5c82b6f7-f547-4a72-8c52-8d7462f0a63d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d2aa732961d4d9790b8543368c290b9', 'request_id': 'req-5c82b6f7-f547-4a72-8c52-8d7462f0a63d'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:21:22.926457 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-5c82b6f7-f547-4a72-8c52-8d7462f0a63d admin admin] The token cache is being invalidate because project 5d2aa732961d4d9790b8543368c290b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 19:21:22.926772 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-5c82b6f7-f547-4a72-8c52-8d7462f0a63d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:21:22.927408 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-5c82b6f7-f547-4a72-8c52-8d7462f0a63d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:21:22.928868 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1146/2295] 10.222.0.61 () {58 vars in 1233 bytes} [Mon Aug 22 19:21:22 2022] DELETE /identity/v3/projects/5d2aa732961d4d9790b8543368c290b9 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:21:22.936701 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b03d3fb-eb4f-4987-b7e4-b032dbcd86e3 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:21:22.937109 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b03d3fb-eb4f-4987-b7e4-b032dbcd86e3 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:21:22.937262 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b03d3fb-eb4f-4987-b7e4-b032dbcd86e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:21:22.956927 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-3b03d3fb-eb4f-4987-b7e4-b032dbcd86e3 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:21:22.958118 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-3b03d3fb-eb4f-4987-b7e4-b032dbcd86e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:21:23.024395 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1150/2296] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:21:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 19:21:23.032739 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1426ee3f-2763-497d-b1d1-6858a9b9bf53 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:21:23.033050 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1426ee3f-2763-497d-b1d1-6858a9b9bf53 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:21:23.033385 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1426ee3f-2763-497d-b1d1-6858a9b9bf53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:21:23.062455 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-1426ee3f-2763-497d-b1d1-6858a9b9bf53 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:21:23.063824 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-1426ee3f-2763-497d-b1d1-6858a9b9bf53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:21:23.071866 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1147/2297] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:21:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:21:23.077324 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b03d3fb-eb4f-4987-b7e4-b032dbcd86e3 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:21:23.087851 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b03d3fb-eb4f-4987-b7e4-b032dbcd86e3 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:21:23.092521 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7497cf7b-52bc-495f-bd3f-3b5bdcc9714d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:21:23.093183 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7497cf7b-52bc-495f-bd3f-3b5bdcc9714d admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:21:23.093461 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7497cf7b-52bc-495f-bd3f-3b5bdcc9714d admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:21:23.093735 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7497cf7b-52bc-495f-bd3f-3b5bdcc9714d admin admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:21:23.094408 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7497cf7b-52bc-495f-bd3f-3b5bdcc9714d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:21:23.094702 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7497cf7b-52bc-495f-bd3f-3b5bdcc9714d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:21:23.096411 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7497cf7b-52bc-495f-bd3f-3b5bdcc9714d admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:21:23.108329 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1151/2298] 10.222.0.61 () {58 vars in 1153 bytes} [Mon Aug 22 19:21:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:21:23.112766 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1426ee3f-2763-497d-b1d1-6858a9b9bf53 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:21:23.123209 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1426ee3f-2763-497d-b1d1-6858a9b9bf53 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:21:23.128313 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f3cd12-50c9-4738-8c26-ef771528dfb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:21:23.128898 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f3cd12-50c9-4738-8c26-ef771528dfb2 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:21:23.129169 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f3cd12-50c9-4738-8c26-ef771528dfb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:21:23.129435 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f3cd12-50c9-4738-8c26-ef771528dfb2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:21:23.130090 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f3cd12-50c9-4738-8c26-ef771528dfb2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:21:23.132011 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f3cd12-50c9-4738-8c26-ef771528dfb2 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:21:23.162028 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1148/2299] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 19:21:23 2022] POST /identity/v3/projects => generated 369 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:21:23.166228 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7497cf7b-52bc-495f-bd3f-3b5bdcc9714d admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:21:23.176540 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7497cf7b-52bc-495f-bd3f-3b5bdcc9714d admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:21:23.241314 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b3b100-d2a4-4e4f-b7dd-f5e688adaab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:21:23.241926 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b3b100-d2a4-4e4f-b7dd-f5e688adaab1 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:21:23.242201 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b3b100-d2a4-4e4f-b7dd-f5e688adaab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:21:23.242470 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b3b100-d2a4-4e4f-b7dd-f5e688adaab1 admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:21:23.243097 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b3b100-d2a4-4e4f-b7dd-f5e688adaab1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:21:23.245577 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b3b100-d2a4-4e4f-b7dd-f5e688adaab1 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:21:23.271723 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1152/2300] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 19:21:23 2022] POST /identity/v3/users => generated 285 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:21:23.272939 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f3cd12-50c9-4738-8c26-ef771528dfb2 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:21:23.283905 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f3cd12-50c9-4738-8c26-ef771528dfb2 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:21:23.292666 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47eb7bff-ff53-4afe-af49-118524ef742a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:21:23.293294 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eb7bff-ff53-4afe-af49-118524ef742a admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:21:23.293573 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eb7bff-ff53-4afe-af49-118524ef742a admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:21:23.294661 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eb7bff-ff53-4afe-af49-118524ef742a admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:21:23.295338 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47eb7bff-ff53-4afe-af49-118524ef742a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:21:23.297017 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47eb7bff-ff53-4afe-af49-118524ef742a admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:21:23.309130 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1149/2301] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:21:23 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:21:23.312986 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b3b100-d2a4-4e4f-b7dd-f5e688adaab1 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:21:23.323676 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b3b100-d2a4-4e4f-b7dd-f5e688adaab1 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:21:23.330009 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75fe8f5-7870-4acb-8676-1ab1c60ddefd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:21:23.330707 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75fe8f5-7870-4acb-8676-1ab1c60ddefd admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:21:23.331057 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75fe8f5-7870-4acb-8676-1ab1c60ddefd admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:21:23.331396 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75fe8f5-7870-4acb-8676-1ab1c60ddefd admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:21:23.332153 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75fe8f5-7870-4acb-8676-1ab1c60ddefd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:21:23.333783 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75fe8f5-7870-4acb-8676-1ab1c60ddefd admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:21:23.345951 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1153/2302] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:21:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:21:23.349842 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47eb7bff-ff53-4afe-af49-118524ef742a admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:21:23.363376 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47eb7bff-ff53-4afe-af49-118524ef742a admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:21:23.371323 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c714042-5f1c-4fef-af67-908eb88b779f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:21:23.372256 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c714042-5f1c-4fef-af67-908eb88b779f admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:21:23.372671 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c714042-5f1c-4fef-af67-908eb88b779f admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:21:23.372955 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c714042-5f1c-4fef-af67-908eb88b779f admin admin] PATH_INFO: `/v3/projects/ad941bb034ba4daa83e99bb9b58615ed/users/df5c3351182742a9aacf7915cfaa8452/roles/74a40bc819cb42eea7e966a4b2a016f1` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:21:23.395964 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c714042-5f1c-4fef-af67-908eb88b779f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad941bb034ba4daa83e99bb9b58615ed, user_id=df5c3351182742a9aacf7915cfaa8452, role_id=74a40bc819cb42eea7e966a4b2a016f1)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:21:23.399004 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c714042-5f1c-4fef-af67-908eb88b779f admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:21:23.413056 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1150/2303] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:21:23 2022] PUT /identity/v3/projects/ad941bb034ba4daa83e99bb9b58615ed/users/df5c3351182742a9aacf7915cfaa8452/roles/74a40bc819cb42eea7e966a4b2a016f1 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:21:23.426915 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0418b2-44f7-4f21-8e65-622cf8d405ec None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:21:23.427288 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0418b2-44f7-4f21-8e65-622cf8d405ec None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:21:23.427782 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0418b2-44f7-4f21-8e65-622cf8d405ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:21:23.461099 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-dc0418b2-44f7-4f21-8e65-622cf8d405ec None None] MFA Rules not processed for user `df5c3351182742a9aacf7915cfaa8452`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:21:23.462577 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-dc0418b2-44f7-4f21-8e65-622cf8d405ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:21:23.484582 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:21:23.537948 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1154/2304] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:21:23 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:21:23.577454 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c714042-5f1c-4fef-af67-908eb88b779f admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:21:23.579113 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-2c714042-5f1c-4fef-af67-908eb88b779f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:21:23.640432 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c714042-5f1c-4fef-af67-908eb88b779f admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:21:23.646248 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58769c40-e9d5-4591-9573-862aa57a2573 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d6a79ed44e5396d13cd29ba160ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:21:23.647307 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58769c40-e9d5-4591-9573-862aa57a2573 service glance] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:21:23.647886 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58769c40-e9d5-4591-9573-862aa57a2573 service glance] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:21:23.648399 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58769c40-e9d5-4591-9573-862aa57a2573 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:21:23.660302 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58769c40-e9d5-4591-9573-862aa57a2573 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:21:23.661770 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58769c40-e9d5-4591-9573-862aa57a2573 service glance] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:21:23.671708 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1151/2305] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:21:23 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:21:33.520998 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0418b2-44f7-4f21-8e65-622cf8d405ec None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:21:33.522494 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-dc0418b2-44f7-4f21-8e65-622cf8d405ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:21:33.591666 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0418b2-44f7-4f21-8e65-622cf8d405ec None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:21:33.596746 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8760167f-8b79-4a5a-897e-ad2a208b07be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:21:33.597377 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8760167f-8b79-4a5a-897e-ad2a208b07be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:21:33.597655 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8760167f-8b79-4a5a-897e-ad2a208b07be admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:21:33.597930 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8760167f-8b79-4a5a-897e-ad2a208b07be admin admin] PATH_INFO: `/v3/users/d5e2e66799164a90a5efc14624a85c7c` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:21:33.599057 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8760167f-8b79-4a5a-897e-ad2a208b07be admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5e2e66799164a90a5efc14624a85c7c)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:21:33.601263 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8760167f-8b79-4a5a-897e-ad2a208b07be admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:21:33.650920 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-8760167f-8b79-4a5a-897e-ad2a208b07be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5e2e66799164a90a5efc14624a85c7c'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:21:33.651544 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-8760167f-8b79-4a5a-897e-ad2a208b07be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5e2e66799164a90a5efc14624a85c7c'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:21:33.659833 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-8760167f-8b79-4a5a-897e-ad2a208b07be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5e2e66799164a90a5efc14624a85c7c'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:21:33.667014 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=57,keystone:DELETE=2,keystone:INSERT=2 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:21:33.668439 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-8760167f-8b79-4a5a-897e-ad2a208b07be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5e2e66799164a90a5efc14624a85c7c'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:21:33.682198 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-8760167f-8b79-4a5a-897e-ad2a208b07be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5e2e66799164a90a5efc14624a85c7c'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:21:33.689297 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1155/2306] 10.222.0.61 () {58 vars in 1224 bytes} [Mon Aug 22 19:21:33 2022] DELETE /identity/v3/users/d5e2e66799164a90a5efc14624a85c7c => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:21:33.694043 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a5d4a4-0e75-4664-afdf-e18960eb4c21 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:21:33.694328 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a5d4a4-0e75-4664-afdf-e18960eb4c21 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:21:33.694604 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a5d4a4-0e75-4664-afdf-e18960eb4c21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:21:33.715191 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-e7a5d4a4-0e75-4664-afdf-e18960eb4c21 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:21:33.716455 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-e7a5d4a4-0e75-4664-afdf-e18960eb4c21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:21:33.776449 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1152/2307] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:21:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:21:33.786962 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8760167f-8b79-4a5a-897e-ad2a208b07be admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:21:33.788651 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-8760167f-8b79-4a5a-897e-ad2a208b07be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:21:33.833131 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:21:33.852187 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8760167f-8b79-4a5a-897e-ad2a208b07be admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:21:33.855927 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f8a5f0-eec9-4f1b-b241-3ec668e1ad9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:21:33.856556 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f8a5f0-eec9-4f1b-b241-3ec668e1ad9d service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:21:33.856836 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f8a5f0-eec9-4f1b-b241-3ec668e1ad9d service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:21:33.857110 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f8a5f0-eec9-4f1b-b241-3ec668e1ad9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:21:33.867136 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f8a5f0-eec9-4f1b-b241-3ec668e1ad9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:21:33.868617 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f8a5f0-eec9-4f1b-b241-3ec668e1ad9d service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:21:33.879167 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1156/2308] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:21:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:21:34.481091 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a5d4a4-0e75-4664-afdf-e18960eb4c21 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:21:34.482650 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-e7a5d4a4-0e75-4664-afdf-e18960eb4c21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:21:34.498565 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a5d4a4-0e75-4664-afdf-e18960eb4c21 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:21:34.504658 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0882a18a-5175-43c9-9f19-126ff714a659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:21:34.505315 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0882a18a-5175-43c9-9f19-126ff714a659 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:21:34.505664 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0882a18a-5175-43c9-9f19-126ff714a659 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:21:34.505861 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0882a18a-5175-43c9-9f19-126ff714a659 admin admin] PATH_INFO: `/v3/projects/495fd878c0ee42b38f082ccdcb9ca384` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:21:34.507690 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0882a18a-5175-43c9-9f19-126ff714a659 admin admin] RBAC: Authorizing `identity:delete_project(project_id=495fd878c0ee42b38f082ccdcb9ca384)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:21:34.510789 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0882a18a-5175-43c9-9f19-126ff714a659 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:21:34.567209 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-0882a18a-5175-43c9-9f19-126ff714a659 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '495fd878c0ee42b38f082ccdcb9ca384', 'request_id': 'req-0882a18a-5175-43c9-9f19-126ff714a659'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:21:34.587984 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-0882a18a-5175-43c9-9f19-126ff714a659 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '495fd878c0ee42b38f082ccdcb9ca384', 'request_id': 'req-0882a18a-5175-43c9-9f19-126ff714a659'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:21:34.595668 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-0882a18a-5175-43c9-9f19-126ff714a659 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '495fd878c0ee42b38f082ccdcb9ca384', 'request_id': 'req-0882a18a-5175-43c9-9f19-126ff714a659'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:21:34.612623 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-0882a18a-5175-43c9-9f19-126ff714a659 admin admin] The token cache is being invalidate because project 495fd878c0ee42b38f082ccdcb9ca384 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 19:21:34.612912 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-0882a18a-5175-43c9-9f19-126ff714a659 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:21:34.613609 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-0882a18a-5175-43c9-9f19-126ff714a659 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:21:34.615182 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1153/2309] 10.222.0.61 () {58 vars in 1233 bytes} [Mon Aug 22 19:21:34 2022] DELETE /identity/v3/projects/495fd878c0ee42b38f082ccdcb9ca384 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:21:43.779136 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf00d498-1770-4b88-a326-a65ae7067612 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:21:43.779728 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf00d498-1770-4b88-a326-a65ae7067612 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:21:43.780688 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf00d498-1770-4b88-a326-a65ae7067612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:21:43.801136 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-bf00d498-1770-4b88-a326-a65ae7067612 None None] MFA Rules not processed for user `8030a8ebdd8c45c7bd30403c866a9278`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:21:43.802634 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-bf00d498-1770-4b88-a326-a65ae7067612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:21:43.858408 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1157/2310] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:21:43 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:21:44.609713 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:21:50.004168 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0882a18a-5175-43c9-9f19-126ff714a659 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:21:50.005700 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-0882a18a-5175-43c9-9f19-126ff714a659 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:21:50.073453 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0882a18a-5175-43c9-9f19-126ff714a659 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:21:50.078400 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4907ea-f91b-45c6-9fc6-49abfdd0eace service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eef0f73297e04d62bc22acb790fad6ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:21:50.079003 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4907ea-f91b-45c6-9fc6-49abfdd0eace service placement] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:21:50.079283 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4907ea-f91b-45c6-9fc6-49abfdd0eace service placement] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:21:50.079558 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4907ea-f91b-45c6-9fc6-49abfdd0eace service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:21:50.081798 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-fd4907ea-f91b-45c6-9fc6-49abfdd0eace service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:21:50.093158 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4907ea-f91b-45c6-9fc6-49abfdd0eace service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:21:50.094753 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4907ea-f91b-45c6-9fc6-49abfdd0eace service placement] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:21:50.107734 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1154/2311] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:21:50 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:21:53.851619 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=34 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:22:00.103425 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=32 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:22:10.432560 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b6a1bd-3d7f-4039-8409-d7d917007869 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:22:10.433145 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b6a1bd-3d7f-4039-8409-d7d917007869 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:22:10.433145 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b6a1bd-3d7f-4039-8409-d7d917007869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:22:10.452962 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-11b6a1bd-3d7f-4039-8409-d7d917007869 None None] MFA Rules not processed for user `a1b3705aa22743c49e3ac2f37c7f2084`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:22:10.454173 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-11b6a1bd-3d7f-4039-8409-d7d917007869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:22:10.510850 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1158/2312] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:22:10 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:22:11.053664 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4907ea-f91b-45c6-9fc6-49abfdd0eace service placement] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:22:11.055131 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-fd4907ea-f91b-45c6-9fc6-49abfdd0eace service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:22:11.129630 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4907ea-f91b-45c6-9fc6-49abfdd0eace service placement] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:22:11.134188 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1575822-3bee-481b-81a2-5358654d078f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:22:11.134815 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1575822-3bee-481b-81a2-5358654d078f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:22:11.135094 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1575822-3bee-481b-81a2-5358654d078f admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:22:11.135367 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1575822-3bee-481b-81a2-5358654d078f admin admin] PATH_INFO: `/v3/users/b628635a98d141539faca7618db4b289` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:22:11.136588 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1575822-3bee-481b-81a2-5358654d078f admin admin] RBAC: Authorizing `identity:delete_user(user_id=b628635a98d141539faca7618db4b289)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:22:11.138476 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1575822-3bee-481b-81a2-5358654d078f admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:22:11.192538 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-f1575822-3bee-481b-81a2-5358654d078f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b628635a98d141539faca7618db4b289'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:22:11.193187 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-f1575822-3bee-481b-81a2-5358654d078f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b628635a98d141539faca7618db4b289'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:22:11.200368 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-f1575822-3bee-481b-81a2-5358654d078f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b628635a98d141539faca7618db4b289'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:22:11.207879 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-f1575822-3bee-481b-81a2-5358654d078f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b628635a98d141539faca7618db4b289'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:22:11.220069 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-f1575822-3bee-481b-81a2-5358654d078f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b628635a98d141539faca7618db4b289'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:22:11.229848 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1155/2313] 10.222.0.61 () {58 vars in 1224 bytes} [Mon Aug 22 19:22:11 2022] DELETE /identity/v3/users/b628635a98d141539faca7618db4b289 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 19:22:11.231580 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82e57f2-1584-471a-90a6-fc94f468fd0f None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:22:11.231905 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82e57f2-1584-471a-90a6-fc94f468fd0f None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:22:11.232184 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82e57f2-1584-471a-90a6-fc94f468fd0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:22:11.252583 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-a82e57f2-1584-471a-90a6-fc94f468fd0f None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:22:11.253756 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-a82e57f2-1584-471a-90a6-fc94f468fd0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:22:11.333029 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1159/2314] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:22:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 19:22:11.342701 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1575822-3bee-481b-81a2-5358654d078f admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:22:11.344420 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-f1575822-3bee-481b-81a2-5358654d078f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:22:11.405482 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1575822-3bee-481b-81a2-5358654d078f admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:22:11.409396 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e11a9e-0a48-4934-a9b6-233cfc313dd5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:22:11.410013 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e11a9e-0a48-4934-a9b6-233cfc313dd5 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:22:11.410347 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e11a9e-0a48-4934-a9b6-233cfc313dd5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:22:11.410625 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e11a9e-0a48-4934-a9b6-233cfc313dd5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:22:11.421072 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e11a9e-0a48-4934-a9b6-233cfc313dd5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:22:11.422479 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e11a9e-0a48-4934-a9b6-233cfc313dd5 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:22:11.432439 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1156/2315] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:22:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:22:11.740159 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82e57f2-1584-471a-90a6-fc94f468fd0f None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:22:11.742074 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-a82e57f2-1584-471a-90a6-fc94f468fd0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:22:11.757931 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82e57f2-1584-471a-90a6-fc94f468fd0f None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:22:11.762961 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551594e9-0c87-4d10-bf80-a14e64f4453b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:22:11.763801 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551594e9-0c87-4d10-bf80-a14e64f4453b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:22:11.764265 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551594e9-0c87-4d10-bf80-a14e64f4453b admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:22:11.764704 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551594e9-0c87-4d10-bf80-a14e64f4453b admin admin] PATH_INFO: `/v3/projects/1c36c32a540942ccbd0ab0ecbce31a58` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:22:11.766557 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551594e9-0c87-4d10-bf80-a14e64f4453b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c36c32a540942ccbd0ab0ecbce31a58)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:22:11.768814 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551594e9-0c87-4d10-bf80-a14e64f4453b admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:22:11.827025 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-551594e9-0c87-4d10-bf80-a14e64f4453b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c36c32a540942ccbd0ab0ecbce31a58', 'request_id': 'req-551594e9-0c87-4d10-bf80-a14e64f4453b'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:22:11.835042 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-551594e9-0c87-4d10-bf80-a14e64f4453b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c36c32a540942ccbd0ab0ecbce31a58', 'request_id': 'req-551594e9-0c87-4d10-bf80-a14e64f4453b'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:22:11.841589 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-551594e9-0c87-4d10-bf80-a14e64f4453b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c36c32a540942ccbd0ab0ecbce31a58', 'request_id': 'req-551594e9-0c87-4d10-bf80-a14e64f4453b'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:22:11.864777 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-551594e9-0c87-4d10-bf80-a14e64f4453b admin admin] The token cache is being invalidate because project 1c36c32a540942ccbd0ab0ecbce31a58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 19:22:11.865363 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-551594e9-0c87-4d10-bf80-a14e64f4453b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:22:11.866487 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-551594e9-0c87-4d10-bf80-a14e64f4453b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:22:11.869367 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1160/2316] 10.222.0.61 () {58 vars in 1233 bytes} [Mon Aug 22 19:22:11 2022] DELETE /identity/v3/projects/1c36c32a540942ccbd0ab0ecbce31a58 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:22:11.873799 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0052f671-ae0a-42b5-a7dc-175a8f1accc3 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:22:11.874226 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0052f671-ae0a-42b5-a7dc-175a8f1accc3 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:22:11.874503 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0052f671-ae0a-42b5-a7dc-175a8f1accc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:22:11.897282 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-0052f671-ae0a-42b5-a7dc-175a8f1accc3 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:22:11.898595 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-0052f671-ae0a-42b5-a7dc-175a8f1accc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:22:11.924332 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:22:11.980243 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1157/2317] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:22:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:22:11.989908 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf55a8e9-aa7c-403a-9e66-456b881ef473 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:22:11.990191 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf55a8e9-aa7c-403a-9e66-456b881ef473 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:22:11.990462 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf55a8e9-aa7c-403a-9e66-456b881ef473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:22:12.010561 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-bf55a8e9-aa7c-403a-9e66-456b881ef473 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:22:12.011813 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-bf55a8e9-aa7c-403a-9e66-456b881ef473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:22:12.019723 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1161/2318] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:22:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 19:22:12.023755 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0052f671-ae0a-42b5-a7dc-175a8f1accc3 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:22:12.033612 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0052f671-ae0a-42b5-a7dc-175a8f1accc3 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:22:12.038221 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6ca26d-6aec-4462-8e05-87a356d5fadf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:22:12.038815 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6ca26d-6aec-4462-8e05-87a356d5fadf admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:22:12.039092 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6ca26d-6aec-4462-8e05-87a356d5fadf admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:22:12.039363 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6ca26d-6aec-4462-8e05-87a356d5fadf admin admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:22:12.040058 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6ca26d-6aec-4462-8e05-87a356d5fadf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:22:12.040428 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6ca26d-6aec-4462-8e05-87a356d5fadf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:22:12.042011 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6ca26d-6aec-4462-8e05-87a356d5fadf admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:22:12.053988 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1158/2319] 10.222.0.61 () {58 vars in 1153 bytes} [Mon Aug 22 19:22:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:22:12.058209 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf55a8e9-aa7c-403a-9e66-456b881ef473 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:22:12.068343 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf55a8e9-aa7c-403a-9e66-456b881ef473 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:22:12.073176 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1b2391-d1f3-49e6-b498-9c1f4b1aa9b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:22:12.073800 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1b2391-d1f3-49e6-b498-9c1f4b1aa9b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:22:12.074078 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1b2391-d1f3-49e6-b498-9c1f4b1aa9b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:22:12.074349 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1b2391-d1f3-49e6-b498-9c1f4b1aa9b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:22:12.075010 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1b2391-d1f3-49e6-b498-9c1f4b1aa9b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:22:12.076937 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1b2391-d1f3-49e6-b498-9c1f4b1aa9b2 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:22:12.097584 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:22:12.105389 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1162/2320] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 19:22:12 2022] POST /identity/v3/projects => generated 381 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:22:12.109522 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6ca26d-6aec-4462-8e05-87a356d5fadf admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:22:12.120391 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6ca26d-6aec-4462-8e05-87a356d5fadf admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:22:12.175073 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35f1ea0-b36a-42df-84e4-9ffe1b3ab5e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:22:12.175693 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35f1ea0-b36a-42df-84e4-9ffe1b3ab5e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:22:12.176572 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35f1ea0-b36a-42df-84e4-9ffe1b3ab5e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:22:12.176848 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35f1ea0-b36a-42df-84e4-9ffe1b3ab5e1 admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:22:12.177467 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35f1ea0-b36a-42df-84e4-9ffe1b3ab5e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:22:12.179353 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35f1ea0-b36a-42df-84e4-9ffe1b3ab5e1 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:22:12.199405 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1159/2321] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 19:22:12 2022] POST /identity/v3/users => generated 291 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 19:22:12.203697 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1b2391-d1f3-49e6-b498-9c1f4b1aa9b2 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:22:12.213971 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1b2391-d1f3-49e6-b498-9c1f4b1aa9b2 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:22:12.218631 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed02ff0-b529-4ab8-90e0-fa419d2938be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:22:12.219224 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed02ff0-b529-4ab8-90e0-fa419d2938be admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:22:12.219504 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed02ff0-b529-4ab8-90e0-fa419d2938be admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:22:12.219797 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed02ff0-b529-4ab8-90e0-fa419d2938be admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:22:12.220439 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed02ff0-b529-4ab8-90e0-fa419d2938be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:22:12.222045 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed02ff0-b529-4ab8-90e0-fa419d2938be admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:22:12.233132 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1163/2322] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:22:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:22:12.237083 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35f1ea0-b36a-42df-84e4-9ffe1b3ab5e1 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:22:12.247716 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35f1ea0-b36a-42df-84e4-9ffe1b3ab5e1 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:22:12.252288 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec7e32c-b29e-43c0-b2fa-c432dd74ef0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:22:12.253199 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec7e32c-b29e-43c0-b2fa-c432dd74ef0f admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:22:12.253483 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec7e32c-b29e-43c0-b2fa-c432dd74ef0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:22:12.253758 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec7e32c-b29e-43c0-b2fa-c432dd74ef0f admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:22:12.254403 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec7e32c-b29e-43c0-b2fa-c432dd74ef0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:22:12.256041 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec7e32c-b29e-43c0-b2fa-c432dd74ef0f admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:22:12.267618 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1160/2323] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:22:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:22:12.271725 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed02ff0-b529-4ab8-90e0-fa419d2938be admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:22:12.281644 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed02ff0-b529-4ab8-90e0-fa419d2938be admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:22:12.288345 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ac68fe-915a-490c-b3ed-33fac8e504ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:22:12.289022 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ac68fe-915a-490c-b3ed-33fac8e504ba admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:22:12.289329 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ac68fe-915a-490c-b3ed-33fac8e504ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:22:12.289601 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ac68fe-915a-490c-b3ed-33fac8e504ba admin admin] PATH_INFO: `/v3/projects/44cc7501b8ad41cc93825b4480d7c3df/users/aa9a0d600b1c42ec8fbb6710744bbd1d/roles/74a40bc819cb42eea7e966a4b2a016f1` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:22:12.305244 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ac68fe-915a-490c-b3ed-33fac8e504ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=44cc7501b8ad41cc93825b4480d7c3df, user_id=aa9a0d600b1c42ec8fbb6710744bbd1d, role_id=74a40bc819cb42eea7e966a4b2a016f1)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:22:12.308183 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ac68fe-915a-490c-b3ed-33fac8e504ba admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:22:12.322021 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1164/2324] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:22:12 2022] PUT /identity/v3/projects/44cc7501b8ad41cc93825b4480d7c3df/users/aa9a0d600b1c42ec8fbb6710744bbd1d/roles/74a40bc819cb42eea7e966a4b2a016f1 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:22:12.327299 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58ecf68-4fa2-4c3c-9f48-6bb0e39a3140 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:22:12.327632 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58ecf68-4fa2-4c3c-9f48-6bb0e39a3140 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:22:12.327937 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58ecf68-4fa2-4c3c-9f48-6bb0e39a3140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:22:12.348610 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-d58ecf68-4fa2-4c3c-9f48-6bb0e39a3140 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:22:12.350081 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-d58ecf68-4fa2-4c3c-9f48-6bb0e39a3140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:22:12.423571 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1161/2325] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:22:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:22:12.433645 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ac68fe-915a-490c-b3ed-33fac8e504ba admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:22:12.435675 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-c7ac68fe-915a-490c-b3ed-33fac8e504ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:22:12.502210 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ac68fe-915a-490c-b3ed-33fac8e504ba admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:22:12.506063 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fc5bac-59cd-4e52-bffa-46d2ccf6a881 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:22:12.506601 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fc5bac-59cd-4e52-bffa-46d2ccf6a881 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:22:12.506894 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fc5bac-59cd-4e52-bffa-46d2ccf6a881 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:22:12.507164 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fc5bac-59cd-4e52-bffa-46d2ccf6a881 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:22:12.516497 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fc5bac-59cd-4e52-bffa-46d2ccf6a881 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:22:12.517916 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fc5bac-59cd-4e52-bffa-46d2ccf6a881 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:22:12.531617 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1165/2326] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:22:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:22:13.199371 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a19a0f-fab2-4c1e-be80-8bd64ab3e50e None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:22:13.199944 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a19a0f-fab2-4c1e-be80-8bd64ab3e50e None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:22:13.199944 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a19a0f-fab2-4c1e-be80-8bd64ab3e50e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:22:13.231328 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-03a19a0f-fab2-4c1e-be80-8bd64ab3e50e None None] MFA Rules not processed for user `aa9a0d600b1c42ec8fbb6710744bbd1d`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:22:13.232591 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-03a19a0f-fab2-4c1e-be80-8bd64ab3e50e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:22:13.252350 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:22:13.317395 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1162/2327] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:22:13 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:22:13.329039 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fc5bac-59cd-4e52-bffa-46d2ccf6a881 service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:22:13.332102 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-d2fc5bac-59cd-4e52-bffa-46d2ccf6a881 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:22:13.411966 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fc5bac-59cd-4e52-bffa-46d2ccf6a881 service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:22:13.416090 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8064d150-2223-45c4-a5c0-f54d534a6a65 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e79498873d4ed29a0e19bbea9b9214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:22:13.416787 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8064d150-2223-45c4-a5c0-f54d534a6a65 service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:22:13.417132 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8064d150-2223-45c4-a5c0-f54d534a6a65 service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:22:13.417500 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8064d150-2223-45c4-a5c0-f54d534a6a65 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:22:13.427798 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8064d150-2223-45c4-a5c0-f54d534a6a65 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:22:13.429280 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8064d150-2223-45c4-a5c0-f54d534a6a65 service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:22:13.439225 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1166/2328] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:22:13 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:22:21.132229 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1481b840-86dc-45ff-8ae2-19b9efc9266a None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:22:21.132724 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1481b840-86dc-45ff-8ae2-19b9efc9266a None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:22:21.132837 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1481b840-86dc-45ff-8ae2-19b9efc9266a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:22:21.151994 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-1481b840-86dc-45ff-8ae2-19b9efc9266a None None] MFA Rules not processed for user `aa9a0d600b1c42ec8fbb6710744bbd1d`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:22:21.153201 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-1481b840-86dc-45ff-8ae2-19b9efc9266a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:22:21.160039 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1163/2329] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:22:21 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:22:23.434548 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=67,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:22:31.147347 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=22 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:22:36.100961 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8064d150-2223-45c4-a5c0-f54d534a6a65 service cinder] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:22:36.102387 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-8064d150-2223-45c4-a5c0-f54d534a6a65 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:22:36.116956 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8064d150-2223-45c4-a5c0-f54d534a6a65 service cinder] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:22:36.121555 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b37c87-308f-4ea4-83f1-36ae3d898a7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:22:36.122179 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b37c87-308f-4ea4-83f1-36ae3d898a7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:22:36.122458 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b37c87-308f-4ea4-83f1-36ae3d898a7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:22:36.122732 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b37c87-308f-4ea4-83f1-36ae3d898a7a admin admin] PATH_INFO: `/v3/users/df5c3351182742a9aacf7915cfaa8452` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:22:36.123891 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b37c87-308f-4ea4-83f1-36ae3d898a7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=df5c3351182742a9aacf7915cfaa8452)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:22:36.125807 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b37c87-308f-4ea4-83f1-36ae3d898a7a admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:22:36.170877 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-24b37c87-308f-4ea4-83f1-36ae3d898a7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df5c3351182742a9aacf7915cfaa8452'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:22:36.171475 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-24b37c87-308f-4ea4-83f1-36ae3d898a7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df5c3351182742a9aacf7915cfaa8452'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:22:36.178999 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-24b37c87-308f-4ea4-83f1-36ae3d898a7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df5c3351182742a9aacf7915cfaa8452'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:22:36.186302 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-24b37c87-308f-4ea4-83f1-36ae3d898a7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df5c3351182742a9aacf7915cfaa8452'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:22:36.198333 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-24b37c87-308f-4ea4-83f1-36ae3d898a7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df5c3351182742a9aacf7915cfaa8452'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:22:36.204154 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1167/2330] 10.222.0.61 () {58 vars in 1224 bytes} [Mon Aug 22 19:22:36 2022] DELETE /identity/v3/users/df5c3351182742a9aacf7915cfaa8452 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:22:36.208698 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7801cc-322e-40a6-b3d4-1a2ec79955f7 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:22:36.209153 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7801cc-322e-40a6-b3d4-1a2ec79955f7 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:22:36.209274 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7801cc-322e-40a6-b3d4-1a2ec79955f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:22:36.228262 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-cb7801cc-322e-40a6-b3d4-1a2ec79955f7 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:22:36.229489 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-cb7801cc-322e-40a6-b3d4-1a2ec79955f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:22:36.287073 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1164/2331] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:22:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:22:36.296640 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b37c87-308f-4ea4-83f1-36ae3d898a7a admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:22:36.298017 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-24b37c87-308f-4ea4-83f1-36ae3d898a7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:22:36.355736 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b37c87-308f-4ea4-83f1-36ae3d898a7a admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:22:36.359382 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c2f6b1-aae9-41f9-aab7-cb7565589093 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:22:36.360017 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c2f6b1-aae9-41f9-aab7-cb7565589093 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:22:36.360298 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c2f6b1-aae9-41f9-aab7-cb7565589093 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:22:36.360573 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c2f6b1-aae9-41f9-aab7-cb7565589093 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:22:36.370163 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c2f6b1-aae9-41f9-aab7-cb7565589093 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:22:36.371586 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c2f6b1-aae9-41f9-aab7-cb7565589093 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:22:36.381076 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1168/2332] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:22:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:22:36.647831 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7801cc-322e-40a6-b3d4-1a2ec79955f7 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:22:36.649265 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-cb7801cc-322e-40a6-b3d4-1a2ec79955f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:22:36.661822 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7801cc-322e-40a6-b3d4-1a2ec79955f7 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:22:36.666750 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcc69a9-eeb7-482c-9b32-8ce9c24a8381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:22:36.667395 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcc69a9-eeb7-482c-9b32-8ce9c24a8381 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:22:36.667704 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcc69a9-eeb7-482c-9b32-8ce9c24a8381 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:22:36.667978 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcc69a9-eeb7-482c-9b32-8ce9c24a8381 admin admin] PATH_INFO: `/v3/projects/ad941bb034ba4daa83e99bb9b58615ed` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:22:36.669232 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcc69a9-eeb7-482c-9b32-8ce9c24a8381 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad941bb034ba4daa83e99bb9b58615ed)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:22:36.671203 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcc69a9-eeb7-482c-9b32-8ce9c24a8381 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:22:36.717960 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-ffcc69a9-eeb7-482c-9b32-8ce9c24a8381 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad941bb034ba4daa83e99bb9b58615ed', 'request_id': 'req-ffcc69a9-eeb7-482c-9b32-8ce9c24a8381'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:22:36.726030 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-ffcc69a9-eeb7-482c-9b32-8ce9c24a8381 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad941bb034ba4daa83e99bb9b58615ed', 'request_id': 'req-ffcc69a9-eeb7-482c-9b32-8ce9c24a8381'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:22:36.732237 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-ffcc69a9-eeb7-482c-9b32-8ce9c24a8381 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad941bb034ba4daa83e99bb9b58615ed', 'request_id': 'req-ffcc69a9-eeb7-482c-9b32-8ce9c24a8381'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:22:36.742342 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-ffcc69a9-eeb7-482c-9b32-8ce9c24a8381 admin admin] The token cache is being invalidate because project ad941bb034ba4daa83e99bb9b58615ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 19:22:36.742624 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-ffcc69a9-eeb7-482c-9b32-8ce9c24a8381 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:22:36.743240 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-ffcc69a9-eeb7-482c-9b32-8ce9c24a8381 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:22:36.744730 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1165/2333] 10.222.0.61 () {58 vars in 1233 bytes} [Mon Aug 22 19:22:36 2022] DELETE /identity/v3/projects/ad941bb034ba4daa83e99bb9b58615ed => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:22:36.749606 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a4155f-0ec6-49be-8b98-6dc166b169a4 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:22:36.749899 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a4155f-0ec6-49be-8b98-6dc166b169a4 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:22:36.750178 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a4155f-0ec6-49be-8b98-6dc166b169a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:22:36.769177 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-10a4155f-0ec6-49be-8b98-6dc166b169a4 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:22:36.770897 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-10a4155f-0ec6-49be-8b98-6dc166b169a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:22:36.831604 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1169/2334] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:22:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:22:36.840413 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1831cf-ebb0-4d82-b243-ed710df1adad None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:22:36.840593 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1831cf-ebb0-4d82-b243-ed710df1adad None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:22:36.840785 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1831cf-ebb0-4d82-b243-ed710df1adad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:22:36.861643 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-dd1831cf-ebb0-4d82-b243-ed710df1adad None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:22:36.862878 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-dd1831cf-ebb0-4d82-b243-ed710df1adad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:22:36.874079 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1166/2335] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:22:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:22:36.876062 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a4155f-0ec6-49be-8b98-6dc166b169a4 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:22:36.887097 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a4155f-0ec6-49be-8b98-6dc166b169a4 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:22:36.891978 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd218d6-758a-4a43-9aa1-8a803d33a1a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:22:36.892587 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd218d6-758a-4a43-9aa1-8a803d33a1a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:22:36.892870 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd218d6-758a-4a43-9aa1-8a803d33a1a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:22:36.893145 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd218d6-758a-4a43-9aa1-8a803d33a1a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:22:36.893800 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd218d6-758a-4a43-9aa1-8a803d33a1a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:22:36.894195 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd218d6-758a-4a43-9aa1-8a803d33a1a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:22:36.896487 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd218d6-758a-4a43-9aa1-8a803d33a1a7 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:22:36.909264 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1170/2336] 10.222.0.61 () {58 vars in 1153 bytes} [Mon Aug 22 19:22:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:22:36.914041 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1831cf-ebb0-4d82-b243-ed710df1adad None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:22:36.924347 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1831cf-ebb0-4d82-b243-ed710df1adad None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:22:36.929724 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44b7038-abdd-47f6-9fd3-671ea742d180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:22:36.930388 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44b7038-abdd-47f6-9fd3-671ea742d180 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:22:36.930666 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44b7038-abdd-47f6-9fd3-671ea742d180 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:22:36.930933 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44b7038-abdd-47f6-9fd3-671ea742d180 admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:22:36.931584 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44b7038-abdd-47f6-9fd3-671ea742d180 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:22:36.933498 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44b7038-abdd-47f6-9fd3-671ea742d180 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:22:36.961436 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1167/2337] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 19:22:36 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:22:36.966233 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd218d6-758a-4a43-9aa1-8a803d33a1a7 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:22:36.971782 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:22:36.978777 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd218d6-758a-4a43-9aa1-8a803d33a1a7 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:22:37.047503 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485f122a-1b9a-4168-ae74-8d6cbdab9f3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:22:37.048176 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485f122a-1b9a-4168-ae74-8d6cbdab9f3f admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:22:37.048461 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485f122a-1b9a-4168-ae74-8d6cbdab9f3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:22:37.048736 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485f122a-1b9a-4168-ae74-8d6cbdab9f3f admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:22:37.050045 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485f122a-1b9a-4168-ae74-8d6cbdab9f3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:22:37.051638 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485f122a-1b9a-4168-ae74-8d6cbdab9f3f admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:22:37.080298 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1171/2338] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 19:22:36 2022] POST /identity/v3/users => generated 288 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:22:37.081665 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44b7038-abdd-47f6-9fd3-671ea742d180 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:22:37.095367 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44b7038-abdd-47f6-9fd3-671ea742d180 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:22:37.122357 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0250a6b-321e-41f2-a0fa-46cc6c19940a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:22:37.124559 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0250a6b-321e-41f2-a0fa-46cc6c19940a admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:22:37.124846 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0250a6b-321e-41f2-a0fa-46cc6c19940a admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:22:37.131111 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0250a6b-321e-41f2-a0fa-46cc6c19940a admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:22:37.131679 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0250a6b-321e-41f2-a0fa-46cc6c19940a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:22:37.133461 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0250a6b-321e-41f2-a0fa-46cc6c19940a admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:22:37.155761 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1168/2339] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:22:37 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:22:37.159832 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485f122a-1b9a-4168-ae74-8d6cbdab9f3f admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:22:37.181194 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485f122a-1b9a-4168-ae74-8d6cbdab9f3f admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:22:37.196844 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d18f71-9faa-4249-8613-4ece6d5880a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:22:37.197472 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d18f71-9faa-4249-8613-4ece6d5880a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:22:37.197751 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d18f71-9faa-4249-8613-4ece6d5880a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:22:37.198026 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d18f71-9faa-4249-8613-4ece6d5880a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:22:37.199415 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d18f71-9faa-4249-8613-4ece6d5880a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:22:37.201238 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d18f71-9faa-4249-8613-4ece6d5880a7 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:22:37.217345 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1172/2340] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:22:37 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:22:37.221317 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0250a6b-321e-41f2-a0fa-46cc6c19940a admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:22:37.233228 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0250a6b-321e-41f2-a0fa-46cc6c19940a admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:22:37.242837 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667d7206-5d96-4847-b31d-5fb8371b18db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:22:37.243514 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667d7206-5d96-4847-b31d-5fb8371b18db admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:22:37.243825 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667d7206-5d96-4847-b31d-5fb8371b18db admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:22:37.244098 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667d7206-5d96-4847-b31d-5fb8371b18db admin admin] PATH_INFO: `/v3/projects/1862e453623341e5bb33e2795aeb073b/users/9f11113f6c964de8943a55aa70cf246e/roles/74a40bc819cb42eea7e966a4b2a016f1` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:22:37.259846 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667d7206-5d96-4847-b31d-5fb8371b18db admin admin] RBAC: Authorizing `identity:create_grant(project_id=1862e453623341e5bb33e2795aeb073b, user_id=9f11113f6c964de8943a55aa70cf246e, role_id=74a40bc819cb42eea7e966a4b2a016f1)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:22:37.262509 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667d7206-5d96-4847-b31d-5fb8371b18db admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:22:37.280902 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1169/2341] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:22:37 2022] PUT /identity/v3/projects/1862e453623341e5bb33e2795aeb073b/users/9f11113f6c964de8943a55aa70cf246e/roles/74a40bc819cb42eea7e966a4b2a016f1 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 19:22:37.292234 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee39cb5-9b8b-4800-bc27-60d6ce577dae None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:22:37.292521 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee39cb5-9b8b-4800-bc27-60d6ce577dae None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:22:37.292826 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee39cb5-9b8b-4800-bc27-60d6ce577dae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:22:37.330889 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-7ee39cb5-9b8b-4800-bc27-60d6ce577dae None None] MFA Rules not processed for user `9f11113f6c964de8943a55aa70cf246e`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:22:37.332411 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-7ee39cb5-9b8b-4800-bc27-60d6ce577dae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:22:37.399929 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1173/2342] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:22:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:22:37.412492 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667d7206-5d96-4847-b31d-5fb8371b18db admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:22:37.414412 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-667d7206-5d96-4847-b31d-5fb8371b18db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:22:37.438179 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:22:37.476652 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667d7206-5d96-4847-b31d-5fb8371b18db admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:22:37.480353 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe074bd-3648-4624-9d26-1c5aac209133 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e79498873d4ed29a0e19bbea9b9214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:22:37.480961 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe074bd-3648-4624-9d26-1c5aac209133 service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:22:37.481238 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe074bd-3648-4624-9d26-1c5aac209133 service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:22:37.481510 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe074bd-3648-4624-9d26-1c5aac209133 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:22:37.491062 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe074bd-3648-4624-9d26-1c5aac209133 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:22:37.492793 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe074bd-3648-4624-9d26-1c5aac209133 service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:22:37.501660 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1170/2343] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:22:37 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:22:43.272178 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee39cb5-9b8b-4800-bc27-60d6ce577dae None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:22:43.273637 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-7ee39cb5-9b8b-4800-bc27-60d6ce577dae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:22:43.335981 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee39cb5-9b8b-4800-bc27-60d6ce577dae None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:22:43.339727 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd7a0f8-67f4-4486-8eb8-7dbb2c8a4f0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:22:43.340368 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd7a0f8-67f4-4486-8eb8-7dbb2c8a4f0a service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:22:43.340643 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd7a0f8-67f4-4486-8eb8-7dbb2c8a4f0a service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:22:43.340913 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd7a0f8-67f4-4486-8eb8-7dbb2c8a4f0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:22:43.343475 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-1bd7a0f8-67f4-4486-8eb8-7dbb2c8a4f0a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:22:43.374095 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:22:43.418528 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd7a0f8-67f4-4486-8eb8-7dbb2c8a4f0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:22:43.420057 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd7a0f8-67f4-4486-8eb8-7dbb2c8a4f0a service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:22:43.430340 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1174/2344] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:22:43 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:22:47.497615 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=17 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:22:48.849578 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe074bd-3648-4624-9d26-1c5aac209133 service cinder] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:22:48.910346 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe074bd-3648-4624-9d26-1c5aac209133 service cinder] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:22:48.914772 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687f90e6-a864-4f87-a73e-67205520a0b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:22:48.915524 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687f90e6-a864-4f87-a73e-67205520a0b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:22:48.915885 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687f90e6-a864-4f87-a73e-67205520a0b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:22:48.916175 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687f90e6-a864-4f87-a73e-67205520a0b3 admin admin] PATH_INFO: `/v3/users/9f11113f6c964de8943a55aa70cf246e` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:22:48.917300 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687f90e6-a864-4f87-a73e-67205520a0b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f11113f6c964de8943a55aa70cf246e)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:22:48.919711 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687f90e6-a864-4f87-a73e-67205520a0b3 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:22:48.971334 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-687f90e6-a864-4f87-a73e-67205520a0b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f11113f6c964de8943a55aa70cf246e'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:22:48.971997 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-687f90e6-a864-4f87-a73e-67205520a0b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f11113f6c964de8943a55aa70cf246e'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:22:48.979237 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-687f90e6-a864-4f87-a73e-67205520a0b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f11113f6c964de8943a55aa70cf246e'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:22:48.986478 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-687f90e6-a864-4f87-a73e-67205520a0b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f11113f6c964de8943a55aa70cf246e'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:22:48.998042 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-687f90e6-a864-4f87-a73e-67205520a0b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f11113f6c964de8943a55aa70cf246e'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:22:49.003760 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1171/2345] 10.222.0.61 () {58 vars in 1224 bytes} [Mon Aug 22 19:22:48 2022] DELETE /identity/v3/users/9f11113f6c964de8943a55aa70cf246e => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:22:49.008657 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea96e361-5758-476f-a25c-272809afcd10 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:22:49.009167 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea96e361-5758-476f-a25c-272809afcd10 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:22:49.009280 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea96e361-5758-476f-a25c-272809afcd10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:22:49.028589 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-ea96e361-5758-476f-a25c-272809afcd10 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:22:49.029823 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-ea96e361-5758-476f-a25c-272809afcd10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:22:49.086707 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1175/2346] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:22:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:22:49.096072 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687f90e6-a864-4f87-a73e-67205520a0b3 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:22:49.097586 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-687f90e6-a864-4f87-a73e-67205520a0b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:22:49.164595 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687f90e6-a864-4f87-a73e-67205520a0b3 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:22:49.168426 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03038573-9f51-4d51-8c5f-5fb52a981ebc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:22:49.169041 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03038573-9f51-4d51-8c5f-5fb52a981ebc service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:22:49.169336 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03038573-9f51-4d51-8c5f-5fb52a981ebc service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:22:49.169593 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03038573-9f51-4d51-8c5f-5fb52a981ebc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:22:49.179182 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03038573-9f51-4d51-8c5f-5fb52a981ebc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:22:49.180645 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03038573-9f51-4d51-8c5f-5fb52a981ebc service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:22:49.190061 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1172/2347] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:22:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:22:49.460975 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea96e361-5758-476f-a25c-272809afcd10 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:22:49.462499 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-ea96e361-5758-476f-a25c-272809afcd10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:22:49.477697 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea96e361-5758-476f-a25c-272809afcd10 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:22:49.482420 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ff6c25-67ca-4e71-9067-b12ddd7587ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:22:49.483062 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ff6c25-67ca-4e71-9067-b12ddd7587ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:22:49.483367 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ff6c25-67ca-4e71-9067-b12ddd7587ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:22:49.483664 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ff6c25-67ca-4e71-9067-b12ddd7587ed admin admin] PATH_INFO: `/v3/projects/1862e453623341e5bb33e2795aeb073b` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:22:49.485051 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ff6c25-67ca-4e71-9067-b12ddd7587ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=1862e453623341e5bb33e2795aeb073b)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:22:49.486925 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ff6c25-67ca-4e71-9067-b12ddd7587ed admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:22:49.534943 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-83ff6c25-67ca-4e71-9067-b12ddd7587ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1862e453623341e5bb33e2795aeb073b', 'request_id': 'req-83ff6c25-67ca-4e71-9067-b12ddd7587ed'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:22:49.542759 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-83ff6c25-67ca-4e71-9067-b12ddd7587ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1862e453623341e5bb33e2795aeb073b', 'request_id': 'req-83ff6c25-67ca-4e71-9067-b12ddd7587ed'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:22:49.549628 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-83ff6c25-67ca-4e71-9067-b12ddd7587ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1862e453623341e5bb33e2795aeb073b', 'request_id': 'req-83ff6c25-67ca-4e71-9067-b12ddd7587ed'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:22:49.560683 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-83ff6c25-67ca-4e71-9067-b12ddd7587ed admin admin] The token cache is being invalidate because project 1862e453623341e5bb33e2795aeb073b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 19:22:49.560982 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-83ff6c25-67ca-4e71-9067-b12ddd7587ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:22:49.561636 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-83ff6c25-67ca-4e71-9067-b12ddd7587ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:22:49.563146 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1176/2348] 10.222.0.61 () {58 vars in 1233 bytes} [Mon Aug 22 19:22:49 2022] DELETE /identity/v3/projects/1862e453623341e5bb33e2795aeb073b => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:22:49.568228 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1310243c-fd08-46e6-970f-c0c715f1ed83 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:22:49.568513 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1310243c-fd08-46e6-970f-c0c715f1ed83 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:22:49.568791 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1310243c-fd08-46e6-970f-c0c715f1ed83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:22:49.588996 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-1310243c-fd08-46e6-970f-c0c715f1ed83 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:22:49.590321 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-1310243c-fd08-46e6-970f-c0c715f1ed83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:22:49.607998 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:22:49.654745 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1173/2349] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:22:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:22:49.663283 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1579357f-0aa4-4a93-9c73-08e1af6b1a06 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:22:49.663567 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1579357f-0aa4-4a93-9c73-08e1af6b1a06 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:22:49.663875 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1579357f-0aa4-4a93-9c73-08e1af6b1a06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:22:49.683741 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-1579357f-0aa4-4a93-9c73-08e1af6b1a06 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:22:49.684949 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-1579357f-0aa4-4a93-9c73-08e1af6b1a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:22:49.692757 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1177/2350] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:22:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:22:49.696733 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1310243c-fd08-46e6-970f-c0c715f1ed83 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:22:49.706446 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1310243c-fd08-46e6-970f-c0c715f1ed83 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:22:49.711109 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd88285-8050-4f74-97de-81f234c5f163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:22:49.711724 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd88285-8050-4f74-97de-81f234c5f163 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:22:49.712008 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd88285-8050-4f74-97de-81f234c5f163 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:22:49.712277 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd88285-8050-4f74-97de-81f234c5f163 admin admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:22:49.712919 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd88285-8050-4f74-97de-81f234c5f163 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:22:49.713243 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd88285-8050-4f74-97de-81f234c5f163 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:22:49.714817 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd88285-8050-4f74-97de-81f234c5f163 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:22:49.727257 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1174/2351] 10.222.0.61 () {58 vars in 1153 bytes} [Mon Aug 22 19:22:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:22:49.731378 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1579357f-0aa4-4a93-9c73-08e1af6b1a06 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:22:49.741443 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1579357f-0aa4-4a93-9c73-08e1af6b1a06 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:22:49.746039 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d73f619-77df-49b7-8ded-a7145e8d3da4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:22:49.746660 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d73f619-77df-49b7-8ded-a7145e8d3da4 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:22:49.746942 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d73f619-77df-49b7-8ded-a7145e8d3da4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:22:49.747201 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d73f619-77df-49b7-8ded-a7145e8d3da4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:22:49.747854 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d73f619-77df-49b7-8ded-a7145e8d3da4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:22:49.749752 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d73f619-77df-49b7-8ded-a7145e8d3da4 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:22:49.776659 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1178/2352] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 19:22:49 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 19:22:49.780714 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd88285-8050-4f74-97de-81f234c5f163 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:22:49.791016 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd88285-8050-4f74-97de-81f234c5f163 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:22:49.848345 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e235d00e-48d4-4ded-818c-4df2da3705e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:22:49.849707 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e235d00e-48d4-4ded-818c-4df2da3705e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:22:49.850327 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e235d00e-48d4-4ded-818c-4df2da3705e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:22:49.850327 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e235d00e-48d4-4ded-818c-4df2da3705e7 admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:22:49.850850 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e235d00e-48d4-4ded-818c-4df2da3705e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:22:49.852849 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e235d00e-48d4-4ded-818c-4df2da3705e7 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:22:49.876687 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1175/2353] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 19:22:49 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:22:49.877800 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d73f619-77df-49b7-8ded-a7145e8d3da4 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:22:49.887939 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d73f619-77df-49b7-8ded-a7145e8d3da4 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:22:49.892512 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bec535-3ebb-407b-a9df-ea804d25241e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:22:49.893101 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bec535-3ebb-407b-a9df-ea804d25241e admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:22:49.893381 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bec535-3ebb-407b-a9df-ea804d25241e admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:22:49.893650 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bec535-3ebb-407b-a9df-ea804d25241e admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:22:49.894286 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bec535-3ebb-407b-a9df-ea804d25241e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:22:49.895914 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bec535-3ebb-407b-a9df-ea804d25241e admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:22:49.905570 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1179/2354] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:22:49 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 19:22:49.909355 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e235d00e-48d4-4ded-818c-4df2da3705e7 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:22:49.920368 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e235d00e-48d4-4ded-818c-4df2da3705e7 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:22:49.925333 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6340f0e9-a237-4751-bf74-82deb1dd5545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:22:49.925956 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6340f0e9-a237-4751-bf74-82deb1dd5545 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:22:49.926233 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6340f0e9-a237-4751-bf74-82deb1dd5545 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:22:49.926506 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6340f0e9-a237-4751-bf74-82deb1dd5545 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:22:49.927156 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6340f0e9-a237-4751-bf74-82deb1dd5545 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:22:49.929484 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6340f0e9-a237-4751-bf74-82deb1dd5545 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:22:49.940707 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1176/2355] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:22:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:22:49.945017 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bec535-3ebb-407b-a9df-ea804d25241e admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:22:49.951753 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:22:49.956330 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bec535-3ebb-407b-a9df-ea804d25241e admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:22:49.962961 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e177d5b-236c-4535-980b-96474c76ab19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:22:49.963772 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e177d5b-236c-4535-980b-96474c76ab19 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:22:49.964083 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e177d5b-236c-4535-980b-96474c76ab19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:22:49.964580 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e177d5b-236c-4535-980b-96474c76ab19 admin admin] PATH_INFO: `/v3/projects/ec13a665e661424fa8602f7fbd962e16/users/cea80d48ea4a467d96ca9dfe2671bdd7/roles/74a40bc819cb42eea7e966a4b2a016f1` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:22:49.980304 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e177d5b-236c-4535-980b-96474c76ab19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec13a665e661424fa8602f7fbd962e16, user_id=cea80d48ea4a467d96ca9dfe2671bdd7, role_id=74a40bc819cb42eea7e966a4b2a016f1)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:22:49.983004 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e177d5b-236c-4535-980b-96474c76ab19 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:22:49.995906 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1180/2356] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:22:49 2022] PUT /identity/v3/projects/ec13a665e661424fa8602f7fbd962e16/users/cea80d48ea4a467d96ca9dfe2671bdd7/roles/74a40bc819cb42eea7e966a4b2a016f1 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:22:50.000873 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bf8464-13c9-4f5b-8c3f-ed0e13fdfe63 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:22:50.001155 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bf8464-13c9-4f5b-8c3f-ed0e13fdfe63 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:22:50.001426 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bf8464-13c9-4f5b-8c3f-ed0e13fdfe63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:22:50.020419 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-13bf8464-13c9-4f5b-8c3f-ed0e13fdfe63 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:22:50.022489 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-13bf8464-13c9-4f5b-8c3f-ed0e13fdfe63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:22:50.085210 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1177/2357] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:22:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:22:50.094841 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e177d5b-236c-4535-980b-96474c76ab19 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:22:50.096597 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-9e177d5b-236c-4535-980b-96474c76ab19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:22:50.157310 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e177d5b-236c-4535-980b-96474c76ab19 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:22:50.161064 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f9a93c-90c9-49b0-b6ce-b10dfeaceda8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:22:50.161674 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f9a93c-90c9-49b0-b6ce-b10dfeaceda8 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:22:50.162566 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f9a93c-90c9-49b0-b6ce-b10dfeaceda8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:22:50.162841 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f9a93c-90c9-49b0-b6ce-b10dfeaceda8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:22:50.172327 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f9a93c-90c9-49b0-b6ce-b10dfeaceda8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:22:50.173751 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f9a93c-90c9-49b0-b6ce-b10dfeaceda8 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:22:50.183363 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1181/2358] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:22:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:22:54.058782 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98b571d-61c2-4fbd-9a06-45d1880eee19 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:22:54.059347 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98b571d-61c2-4fbd-9a06-45d1880eee19 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:22:54.059347 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98b571d-61c2-4fbd-9a06-45d1880eee19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:22:54.093227 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-a98b571d-61c2-4fbd-9a06-45d1880eee19 None None] MFA Rules not processed for user `cea80d48ea4a467d96ca9dfe2671bdd7`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:22:54.094699 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-a98b571d-61c2-4fbd-9a06-45d1880eee19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:22:54.115235 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:22:54.168151 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1178/2359] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:22:54 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:22:54.172545 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f9a93c-90c9-49b0-b6ce-b10dfeaceda8 service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:22:54.182653 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f9a93c-90c9-49b0-b6ce-b10dfeaceda8 service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:22:54.187336 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddd5d34-1c50-46f6-b87f-4c9e03703530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:22:54.188094 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddd5d34-1c50-46f6-b87f-4c9e03703530 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:22:54.188420 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddd5d34-1c50-46f6-b87f-4c9e03703530 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:22:54.188638 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddd5d34-1c50-46f6-b87f-4c9e03703530 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:22:54.189262 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddd5d34-1c50-46f6-b87f-4c9e03703530 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:22:54.191145 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddd5d34-1c50-46f6-b87f-4c9e03703530 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:22:54.221245 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1182/2360] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 19:22:54 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:22:54.222659 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98b571d-61c2-4fbd-9a06-45d1880eee19 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:22:54.235043 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98b571d-61c2-4fbd-9a06-45d1880eee19 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:22:54.309026 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efc0edc-4c58-44fb-a0d6-6cf0e9b4eeb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:22:54.309637 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efc0edc-4c58-44fb-a0d6-6cf0e9b4eeb8 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:22:54.309916 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efc0edc-4c58-44fb-a0d6-6cf0e9b4eeb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:22:54.310187 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efc0edc-4c58-44fb-a0d6-6cf0e9b4eeb8 admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:22:54.310824 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efc0edc-4c58-44fb-a0d6-6cf0e9b4eeb8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:22:54.312837 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efc0edc-4c58-44fb-a0d6-6cf0e9b4eeb8 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:22:54.336144 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1179/2361] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 19:22:54 2022] POST /identity/v3/users => generated 298 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:22:54.340259 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddd5d34-1c50-46f6-b87f-4c9e03703530 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:22:54.351421 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddd5d34-1c50-46f6-b87f-4c9e03703530 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:22:54.356280 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7eb431-1c99-4284-8249-302a847652b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:22:54.356913 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7eb431-1c99-4284-8249-302a847652b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:22:54.357189 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7eb431-1c99-4284-8249-302a847652b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:22:54.357455 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7eb431-1c99-4284-8249-302a847652b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:22:54.358095 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7eb431-1c99-4284-8249-302a847652b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:22:54.359775 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7eb431-1c99-4284-8249-302a847652b7 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:22:54.371977 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1183/2362] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:22:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:22:54.376292 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efc0edc-4c58-44fb-a0d6-6cf0e9b4eeb8 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:22:54.387034 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efc0edc-4c58-44fb-a0d6-6cf0e9b4eeb8 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:22:54.391855 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58460ee-1cb1-46d2-a2ff-fba3a3413f89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:22:54.392511 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58460ee-1cb1-46d2-a2ff-fba3a3413f89 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:22:54.392794 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58460ee-1cb1-46d2-a2ff-fba3a3413f89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:22:54.393061 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58460ee-1cb1-46d2-a2ff-fba3a3413f89 admin admin] PATH_INFO: `/v3/projects/2d1a4c46560346fe99997df6b74808a3/users/466b30e33e4b4000b81ff27b0c3bf26e/roles/163c30c3b3c246beb849ce176f02d9da` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:22:54.408848 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58460ee-1cb1-46d2-a2ff-fba3a3413f89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d1a4c46560346fe99997df6b74808a3, user_id=466b30e33e4b4000b81ff27b0c3bf26e, role_id=163c30c3b3c246beb849ce176f02d9da)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:22:54.411561 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58460ee-1cb1-46d2-a2ff-fba3a3413f89 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:22:54.428826 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1180/2363] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:22:54 2022] PUT /identity/v3/projects/2d1a4c46560346fe99997df6b74808a3/users/466b30e33e4b4000b81ff27b0c3bf26e/roles/163c30c3b3c246beb849ce176f02d9da => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:22:54.429794 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7eb431-1c99-4284-8249-302a847652b7 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:22:54.440295 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7eb431-1c99-4284-8249-302a847652b7 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:22:54.523810 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c193401-20de-4b86-93aa-ff3e58efcf0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:22:54.524552 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c193401-20de-4b86-93aa-ff3e58efcf0e admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:22:54.524898 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c193401-20de-4b86-93aa-ff3e58efcf0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:22:54.525235 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c193401-20de-4b86-93aa-ff3e58efcf0e admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:22:54.526151 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c193401-20de-4b86-93aa-ff3e58efcf0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:22:54.527889 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c193401-20de-4b86-93aa-ff3e58efcf0e admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:22:54.538822 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1184/2364] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:22:54 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 19:22:54.543066 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58460ee-1cb1-46d2-a2ff-fba3a3413f89 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:22:54.554413 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58460ee-1cb1-46d2-a2ff-fba3a3413f89 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:22:54.559327 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84bd2e6-046c-40c7-a859-c7aed7e56fa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:22:54.560023 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84bd2e6-046c-40c7-a859-c7aed7e56fa1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:22:54.560298 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84bd2e6-046c-40c7-a859-c7aed7e56fa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:22:54.560568 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84bd2e6-046c-40c7-a859-c7aed7e56fa1 admin admin] PATH_INFO: `/v3/projects/2d1a4c46560346fe99997df6b74808a3/users/466b30e33e4b4000b81ff27b0c3bf26e/roles/163c30c3b3c246beb849ce176f02d9da` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:22:54.562508 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84bd2e6-046c-40c7-a859-c7aed7e56fa1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d1a4c46560346fe99997df6b74808a3, user_id=466b30e33e4b4000b81ff27b0c3bf26e, role_id=163c30c3b3c246beb849ce176f02d9da)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:22:54.565169 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84bd2e6-046c-40c7-a859-c7aed7e56fa1 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:22:54.576908 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1181/2365] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:22:54 2022] PUT /identity/v3/projects/2d1a4c46560346fe99997df6b74808a3/users/466b30e33e4b4000b81ff27b0c3bf26e/roles/163c30c3b3c246beb849ce176f02d9da => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 19:22:59.307839 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1067c25e-490a-4969-ac1c-dcb8c161e631 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:22:59.308317 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1067c25e-490a-4969-ac1c-dcb8c161e631 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:22:59.308541 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1067c25e-490a-4969-ac1c-dcb8c161e631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:22:59.337258 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-1067c25e-490a-4969-ac1c-dcb8c161e631 None None] MFA Rules not processed for user `466b30e33e4b4000b81ff27b0c3bf26e`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:22:59.338547 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-1067c25e-490a-4969-ac1c-dcb8c161e631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:22:59.372095 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:22:59.423228 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1185/2366] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:22:59 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:22:59.436855 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84bd2e6-046c-40c7-a859-c7aed7e56fa1 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:22:59.440091 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-e84bd2e6-046c-40c7-a859-c7aed7e56fa1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:22:59.504320 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84bd2e6-046c-40c7-a859-c7aed7e56fa1 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:22:59.508503 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178a57b8-b4f9-454c-9b35-44d85b3480c5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d6a79ed44e5396d13cd29ba160ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:22:59.509128 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178a57b8-b4f9-454c-9b35-44d85b3480c5 service glance] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:22:59.509404 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178a57b8-b4f9-454c-9b35-44d85b3480c5 service glance] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:22:59.509676 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178a57b8-b4f9-454c-9b35-44d85b3480c5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:22:59.537804 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:22:59.575870 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178a57b8-b4f9-454c-9b35-44d85b3480c5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:22:59.577620 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178a57b8-b4f9-454c-9b35-44d85b3480c5 service glance] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:22:59.589918 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1182/2367] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:22:59 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:22:59.797498 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7ea1f7-4fee-4868-bbb7-8952a1db6ffb None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:22:59.798234 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7ea1f7-4fee-4868-bbb7-8952a1db6ffb None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:22:59.798234 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7ea1f7-4fee-4868-bbb7-8952a1db6ffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:22:59.824313 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-2d7ea1f7-4fee-4868-bbb7-8952a1db6ffb None None] MFA Rules not processed for user `399396a8fe254da199bc48b0ca5243d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:22:59.825459 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-2d7ea1f7-4fee-4868-bbb7-8952a1db6ffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:22:59.887463 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1186/2368] 10.222.0.61 () {58 vars in 1009 bytes} [Mon Aug 22 19:22:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:22:59.950608 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178a57b8-b4f9-454c-9b35-44d85b3480c5 service glance] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:22:59.952233 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-178a57b8-b4f9-454c-9b35-44d85b3480c5 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:23:00.026577 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178a57b8-b4f9-454c-9b35-44d85b3480c5 service glance] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:23:00.031716 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f161c6-7412-445e-b2ac-c6b873747c06 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2824359491934d0dacaa21a718becee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:23:00.032402 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f161c6-7412-445e-b2ac-c6b873747c06 service swift] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:23:00.032745 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f161c6-7412-445e-b2ac-c6b873747c06 service swift] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:23:00.033083 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f161c6-7412-445e-b2ac-c6b873747c06 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:23:00.042515 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f161c6-7412-445e-b2ac-c6b873747c06 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:23:00.044129 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f161c6-7412-445e-b2ac-c6b873747c06 service swift] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:23:00.052778 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1183/2369] 10.222.0.61 () {60 vars in 1384 bytes} [Mon Aug 22 19:22:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 22 19:23:03.680517 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0862fcb2-852e-4daa-a466-f6097b6b4073 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:23:03.680517 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0862fcb2-852e-4daa-a466-f6097b6b4073 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:23:03.681161 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0862fcb2-852e-4daa-a466-f6097b6b4073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:23:03.705171 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-0862fcb2-852e-4daa-a466-f6097b6b4073 None None] MFA Rules not processed for user `61388384a318473e987ab06426509b69`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:23:03.706433 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-0862fcb2-852e-4daa-a466-f6097b6b4073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:23:03.796015 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1187/2370] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:23:03 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:23:10.049281 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=48 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:23:10.256078 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73afd180-6e57-4ebe-b073-87a1478bdcf1 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:23:10.256363 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73afd180-6e57-4ebe-b073-87a1478bdcf1 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:23:10.256641 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73afd180-6e57-4ebe-b073-87a1478bdcf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:23:10.275416 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-73afd180-6e57-4ebe-b073-87a1478bdcf1 None None] MFA Rules not processed for user `cea80d48ea4a467d96ca9dfe2671bdd7`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:23:10.276698 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-73afd180-6e57-4ebe-b073-87a1478bdcf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:23:10.283472 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1184/2371] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:23:10 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:23:13.783770 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=63 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:23:14.215486 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0862fcb2-852e-4daa-a466-f6097b6b4073 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:23:14.216912 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-0862fcb2-852e-4daa-a466-f6097b6b4073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:23:14.277349 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0862fcb2-852e-4daa-a466-f6097b6b4073 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:23:14.280841 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8893fd85-6f42-48dd-9ac2-92caf1d40593 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:23:14.281440 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8893fd85-6f42-48dd-9ac2-92caf1d40593 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:23:14.282509 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8893fd85-6f42-48dd-9ac2-92caf1d40593 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:23:14.282790 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8893fd85-6f42-48dd-9ac2-92caf1d40593 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:23:14.285181 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-8893fd85-6f42-48dd-9ac2-92caf1d40593 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:23:14.358881 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8893fd85-6f42-48dd-9ac2-92caf1d40593 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:23:14.360298 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8893fd85-6f42-48dd-9ac2-92caf1d40593 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:23:14.371959 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1188/2372] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:23:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:23:14.714248 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc4b6a8-da5f-48d7-818e-dd7b64ef4e51 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:23:14.714736 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc4b6a8-da5f-48d7-818e-dd7b64ef4e51 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:23:14.714845 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc4b6a8-da5f-48d7-818e-dd7b64ef4e51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:23:14.737304 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-2bc4b6a8-da5f-48d7-818e-dd7b64ef4e51 None None] MFA Rules not processed for user `399396a8fe254da199bc48b0ca5243d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:23:14.738506 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-2bc4b6a8-da5f-48d7-818e-dd7b64ef4e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:23:14.745861 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1185/2373] 10.222.0.61 () {58 vars in 1009 bytes} [Mon Aug 22 19:23:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:23:14.757370 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8893fd85-6f42-48dd-9ac2-92caf1d40593 service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:23:14.768572 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8893fd85-6f42-48dd-9ac2-92caf1d40593 service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:23:14.774339 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7d0c07-ab5d-4a53-b719-aec193d32d75 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2824359491934d0dacaa21a718becee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:23:14.775071 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d0c07-ab5d-4a53-b719-aec193d32d75 service swift] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:23:14.775415 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d0c07-ab5d-4a53-b719-aec193d32d75 service swift] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:23:14.776038 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d0c07-ab5d-4a53-b719-aec193d32d75 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:23:14.784481 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7d0c07-ab5d-4a53-b719-aec193d32d75 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:23:14.786831 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7d0c07-ab5d-4a53-b719-aec193d32d75 service swift] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:23:14.795218 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1189/2374] 10.222.0.61 () {60 vars in 1384 bytes} [Mon Aug 22 19:23:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 22 19:23:17.153695 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc4b6a8-da5f-48d7-818e-dd7b64ef4e51 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:23:17.163750 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc4b6a8-da5f-48d7-818e-dd7b64ef4e51 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:23:17.167666 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32879e58-d685-4509-a392-d27032132eda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:23:17.168267 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32879e58-d685-4509-a392-d27032132eda service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:23:17.168545 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32879e58-d685-4509-a392-d27032132eda service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:23:17.168821 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32879e58-d685-4509-a392-d27032132eda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:23:17.170610 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-32879e58-d685-4509-a392-d27032132eda service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:23:17.184202 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32879e58-d685-4509-a392-d27032132eda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:23:17.185656 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32879e58-d685-4509-a392-d27032132eda service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:23:17.196722 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1186/2375] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:23:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:23:23.903144 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7d0c07-ab5d-4a53-b719-aec193d32d75 service swift] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:23:23.904698 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-8c7d0c07-ab5d-4a53-b719-aec193d32d75 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:23:23.921833 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7d0c07-ab5d-4a53-b719-aec193d32d75 service swift] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:23:23.929883 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed6b93e-765b-493a-99a5-17bb3b11431d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f43e2756b746a6825ca1a06b7d7c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:23:23.930768 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed6b93e-765b-493a-99a5-17bb3b11431d service nova] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:23:23.931178 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed6b93e-765b-493a-99a5-17bb3b11431d service nova] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:23:23.931446 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed6b93e-765b-493a-99a5-17bb3b11431d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:23:23.933362 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-7ed6b93e-765b-493a-99a5-17bb3b11431d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:23:23.948753 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed6b93e-765b-493a-99a5-17bb3b11431d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:23:23.950373 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed6b93e-765b-493a-99a5-17bb3b11431d service nova] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:23:23.962083 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1190/2376] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:23:23 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:23:24.256572 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32879e58-d685-4509-a392-d27032132eda service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:23:24.258078 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-32879e58-d685-4509-a392-d27032132eda service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:23:24.271792 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32879e58-d685-4509-a392-d27032132eda service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:23:24.277253 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4560f02-69bd-4cd0-a094-9f6b2005ab7c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f43e2756b746a6825ca1a06b7d7c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:23:24.277982 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4560f02-69bd-4cd0-a094-9f6b2005ab7c service nova] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:23:24.278259 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4560f02-69bd-4cd0-a094-9f6b2005ab7c service nova] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:23:24.278527 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4560f02-69bd-4cd0-a094-9f6b2005ab7c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:23:24.280316 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-a4560f02-69bd-4cd0-a094-9f6b2005ab7c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:23:24.342605 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4560f02-69bd-4cd0-a094-9f6b2005ab7c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:23:24.344164 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4560f02-69bd-4cd0-a094-9f6b2005ab7c service nova] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:23:24.355934 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1187/2377] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:23:24 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:23:33.957242 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=66 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:23:34.351310 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=40 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:23:35.822519 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed6b93e-765b-493a-99a5-17bb3b11431d service nova] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:23:35.825902 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4560f02-69bd-4cd0-a094-9f6b2005ab7c service nova] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:23:35.833752 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed6b93e-765b-493a-99a5-17bb3b11431d service nova] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:23:35.835902 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4560f02-69bd-4cd0-a094-9f6b2005ab7c service nova] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:23:35.837601 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc718f7d-9dd1-4a59-90e6-b0541a248c31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:23:35.838238 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc718f7d-9dd1-4a59-90e6-b0541a248c31 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:23:35.838515 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc718f7d-9dd1-4a59-90e6-b0541a248c31 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:23:35.838788 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc718f7d-9dd1-4a59-90e6-b0541a248c31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:23:35.839957 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77167a95-39fd-4ddf-be71-1edffd16caec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:23:35.840522 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-cc718f7d-9dd1-4a59-90e6-b0541a248c31 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:23:35.840625 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77167a95-39fd-4ddf-be71-1edffd16caec service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:23:35.840849 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77167a95-39fd-4ddf-be71-1edffd16caec service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:23:35.841111 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77167a95-39fd-4ddf-be71-1edffd16caec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:23:35.842843 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-77167a95-39fd-4ddf-be71-1edffd16caec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:23:35.850931 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc718f7d-9dd1-4a59-90e6-b0541a248c31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:23:35.852391 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc718f7d-9dd1-4a59-90e6-b0541a248c31 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:23:35.853179 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77167a95-39fd-4ddf-be71-1edffd16caec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:23:35.854681 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77167a95-39fd-4ddf-be71-1edffd16caec service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:23:35.860313 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1191/2378] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:23:35 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:23:35.862666 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1188/2379] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:23:35 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 19:23:44.249522 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc718f7d-9dd1-4a59-90e6-b0541a248c31 service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:23:44.263064 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc718f7d-9dd1-4a59-90e6-b0541a248c31 service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:23:44.268485 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc0c37b-e90e-4da4-aba2-87c21171c609 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f43e2756b746a6825ca1a06b7d7c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:23:44.269146 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc0c37b-e90e-4da4-aba2-87c21171c609 service nova] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:23:44.269428 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc0c37b-e90e-4da4-aba2-87c21171c609 service nova] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:23:44.269696 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc0c37b-e90e-4da4-aba2-87c21171c609 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:23:44.272438 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-bdc0c37b-e90e-4da4-aba2-87c21171c609 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:23:44.284663 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc0c37b-e90e-4da4-aba2-87c21171c609 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:23:44.286266 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc0c37b-e90e-4da4-aba2-87c21171c609 service nova] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:23:44.297138 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1192/2380] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:23:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:23:45.859115 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=6 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:23:54.293040 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=12 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:23:58.499161 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77167a95-39fd-4ddf-be71-1edffd16caec service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:23:58.512105 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77167a95-39fd-4ddf-be71-1edffd16caec service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:23:58.517856 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62004eff-7334-4334-a9fd-e7d7e35de268 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f43e2756b746a6825ca1a06b7d7c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:23:58.518491 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62004eff-7334-4334-a9fd-e7d7e35de268 service nova] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:23:58.518791 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62004eff-7334-4334-a9fd-e7d7e35de268 service nova] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:23:58.519062 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62004eff-7334-4334-a9fd-e7d7e35de268 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:23:58.521407 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-62004eff-7334-4334-a9fd-e7d7e35de268 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:23:58.533557 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62004eff-7334-4334-a9fd-e7d7e35de268 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:23:58.535125 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62004eff-7334-4334-a9fd-e7d7e35de268 service nova] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:23:58.546112 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1189/2381] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:23:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:24:08.541753 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=6 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:24:12.562743 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8b1425-030f-480b-8100-11075de538f1 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:24:12.563233 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8b1425-030f-480b-8100-11075de538f1 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:24:12.563336 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8b1425-030f-480b-8100-11075de538f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:24:12.582300 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-de8b1425-030f-480b-8100-11075de538f1 None None] MFA Rules not processed for user `aa9a0d600b1c42ec8fbb6710744bbd1d`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:24:12.584180 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-de8b1425-030f-480b-8100-11075de538f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:24:12.638913 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1193/2382] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:24:12 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:24:12.872375 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62004eff-7334-4334-a9fd-e7d7e35de268 service nova] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:24:12.882155 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62004eff-7334-4334-a9fd-e7d7e35de268 service nova] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:24:12.885736 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c77d9d-dd2d-488d-b69d-abbb820bf620 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:24:12.886352 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c77d9d-dd2d-488d-b69d-abbb820bf620 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:24:12.886645 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c77d9d-dd2d-488d-b69d-abbb820bf620 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:24:12.886900 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c77d9d-dd2d-488d-b69d-abbb820bf620 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:24:12.888842 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-20c77d9d-dd2d-488d-b69d-abbb820bf620 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:24:12.898445 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c77d9d-dd2d-488d-b69d-abbb820bf620 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:24:12.899873 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c77d9d-dd2d-488d-b69d-abbb820bf620 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:24:12.907522 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1190/2383] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:24:12 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:24:13.212821 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8b1425-030f-480b-8100-11075de538f1 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:24:13.214209 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-de8b1425-030f-480b-8100-11075de538f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:24:13.297135 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8b1425-030f-480b-8100-11075de538f1 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:24:13.302071 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a9b71e-a71c-42bf-9459-4ce30976b803 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eef0f73297e04d62bc22acb790fad6ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:24:13.303342 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a9b71e-a71c-42bf-9459-4ce30976b803 service placement] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:24:13.303624 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a9b71e-a71c-42bf-9459-4ce30976b803 service placement] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:24:13.303936 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a9b71e-a71c-42bf-9459-4ce30976b803 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:24:13.305683 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-43a9b71e-a71c-42bf-9459-4ce30976b803 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:24:13.318977 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a9b71e-a71c-42bf-9459-4ce30976b803 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:24:13.320661 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a9b71e-a71c-42bf-9459-4ce30976b803 service placement] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:24:13.331456 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1194/2384] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:24:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:24:13.828481 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c77d9d-dd2d-488d-b69d-abbb820bf620 service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:24:13.837641 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c77d9d-dd2d-488d-b69d-abbb820bf620 service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:24:13.841266 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e767c36b-4d8d-4f33-a8bb-0b9e5ba52fc4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:24:13.841856 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e767c36b-4d8d-4f33-a8bb-0b9e5ba52fc4 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:24:13.842127 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e767c36b-4d8d-4f33-a8bb-0b9e5ba52fc4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:24:13.842552 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e767c36b-4d8d-4f33-a8bb-0b9e5ba52fc4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:24:13.852932 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e767c36b-4d8d-4f33-a8bb-0b9e5ba52fc4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:24:13.854373 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e767c36b-4d8d-4f33-a8bb-0b9e5ba52fc4 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:24:13.863572 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1191/2385] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:24:13 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:24:23.327157 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=57 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:24:23.778336 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a9b71e-a71c-42bf-9459-4ce30976b803 service placement] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:24:23.792591 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a9b71e-a71c-42bf-9459-4ce30976b803 service placement] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:24:23.797974 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c707f9b-78bb-4f2f-9daf-0c48a9a81bfb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f43e2756b746a6825ca1a06b7d7c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:24:23.798612 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c707f9b-78bb-4f2f-9daf-0c48a9a81bfb service nova] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:24:23.798885 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c707f9b-78bb-4f2f-9daf-0c48a9a81bfb service nova] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:24:23.799152 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c707f9b-78bb-4f2f-9daf-0c48a9a81bfb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:24:23.800969 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-6c707f9b-78bb-4f2f-9daf-0c48a9a81bfb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:24:23.816736 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c707f9b-78bb-4f2f-9daf-0c48a9a81bfb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:24:23.818349 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c707f9b-78bb-4f2f-9daf-0c48a9a81bfb service nova] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:24:23.829893 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1195/2386] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:24:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:24:23.859479 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=12 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:24:33.825399 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=6 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:24:36.519903 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e767c36b-4d8d-4f33-a8bb-0b9e5ba52fc4 service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:24:36.532239 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e767c36b-4d8d-4f33-a8bb-0b9e5ba52fc4 service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:24:36.538376 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4467611c-7d0f-4565-80c6-a6bf8d139eb9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f43e2756b746a6825ca1a06b7d7c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:24:36.538906 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4467611c-7d0f-4565-80c6-a6bf8d139eb9 service nova] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:24:36.539181 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4467611c-7d0f-4565-80c6-a6bf8d139eb9 service nova] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:24:36.539456 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4467611c-7d0f-4565-80c6-a6bf8d139eb9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:24:36.541233 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-4467611c-7d0f-4565-80c6-a6bf8d139eb9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:24:36.602647 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4467611c-7d0f-4565-80c6-a6bf8d139eb9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:24:36.604268 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4467611c-7d0f-4565-80c6-a6bf8d139eb9 service nova] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:24:36.613300 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1192/2387] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:24:36 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:24:46.609003 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=28 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:26:07.726382 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3840a71e-0a26-47b3-afce-faa6989b5347 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:26:07.726994 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3840a71e-0a26-47b3-afce-faa6989b5347 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:26:07.726994 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3840a71e-0a26-47b3-afce-faa6989b5347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:26:07.747013 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-3840a71e-0a26-47b3-afce-faa6989b5347 None None] MFA Rules not processed for user `8030a8ebdd8c45c7bd30403c866a9278`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:26:07.748266 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-3840a71e-0a26-47b3-afce-faa6989b5347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:26:07.755112 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1196/2388] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:26:07 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:26:13.720357 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4467611c-7d0f-4565-80c6-a6bf8d139eb9 service nova] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:26:13.732188 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4467611c-7d0f-4565-80c6-a6bf8d139eb9 service nova] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:26:13.737540 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f759f6fd-6c10-4e2b-adca-f8dc66281330 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f43e2756b746a6825ca1a06b7d7c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:26:13.738130 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f759f6fd-6c10-4e2b-adca-f8dc66281330 service nova] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:26:13.738405 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f759f6fd-6c10-4e2b-adca-f8dc66281330 service nova] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:26:13.738672 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f759f6fd-6c10-4e2b-adca-f8dc66281330 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:26:13.740381 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-f759f6fd-6c10-4e2b-adca-f8dc66281330 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:26:13.799674 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f759f6fd-6c10-4e2b-adca-f8dc66281330 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:26:13.801251 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f759f6fd-6c10-4e2b-adca-f8dc66281330 service nova] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:26:13.810098 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1193/2389] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:26:13 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:26:17.741159 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:26:23.805983 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=28 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:27:29.252981 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3840a71e-0a26-47b3-afce-faa6989b5347 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:27:29.354228 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3840a71e-0a26-47b3-afce-faa6989b5347 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:27:29.361964 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5b9f18-10d9-4f41-8cbe-0b21907cdada service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eef0f73297e04d62bc22acb790fad6ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:27:29.362708 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5b9f18-10d9-4f41-8cbe-0b21907cdada service placement] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:27:29.363084 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5b9f18-10d9-4f41-8cbe-0b21907cdada service placement] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:27:29.363315 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5b9f18-10d9-4f41-8cbe-0b21907cdada service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:27:29.365280 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-ec5b9f18-10d9-4f41-8cbe-0b21907cdada service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:27:29.415619 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5b9f18-10d9-4f41-8cbe-0b21907cdada service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:27:29.417272 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5b9f18-10d9-4f41-8cbe-0b21907cdada service placement] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:27:29.429967 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1197/2390] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:27:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:27:39.425649 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=36 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:27:40.479713 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f759f6fd-6c10-4e2b-adca-f8dc66281330 service nova] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:27:40.481171 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-f759f6fd-6c10-4e2b-adca-f8dc66281330 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:27:40.576888 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f759f6fd-6c10-4e2b-adca-f8dc66281330 service nova] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:27:40.581462 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab84d00e-1bf3-4774-b40f-b646e89cc066 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e79498873d4ed29a0e19bbea9b9214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:27:40.582056 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab84d00e-1bf3-4774-b40f-b646e89cc066 service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:27:40.582329 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab84d00e-1bf3-4774-b40f-b646e89cc066 service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:27:40.582596 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab84d00e-1bf3-4774-b40f-b646e89cc066 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:27:40.584359 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-ab84d00e-1bf3-4774-b40f-b646e89cc066 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:27:40.595620 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab84d00e-1bf3-4774-b40f-b646e89cc066 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:27:40.597309 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab84d00e-1bf3-4774-b40f-b646e89cc066 service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:27:40.606055 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1194/2391] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:27:40 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:27:50.602102 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=44 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:28:13.417176 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5b9f18-10d9-4f41-8cbe-0b21907cdada service placement] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:28:13.418601 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-ec5b9f18-10d9-4f41-8cbe-0b21907cdada service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:28:13.430394 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5b9f18-10d9-4f41-8cbe-0b21907cdada service placement] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:28:13.433965 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2cee41-46fb-4036-b5f2-c08b4aa0b231 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e79498873d4ed29a0e19bbea9b9214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:28:13.434560 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2cee41-46fb-4036-b5f2-c08b4aa0b231 service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:28:13.434839 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2cee41-46fb-4036-b5f2-c08b4aa0b231 service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:28:13.435109 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2cee41-46fb-4036-b5f2-c08b4aa0b231 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:28:13.452433 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2cee41-46fb-4036-b5f2-c08b4aa0b231 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:28:13.453863 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2cee41-46fb-4036-b5f2-c08b4aa0b231 service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:28:13.477064 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1198/2392] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:28:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:28:13.484175 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab84d00e-1bf3-4774-b40f-b646e89cc066 service cinder] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:28:13.492596 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab84d00e-1bf3-4774-b40f-b646e89cc066 service cinder] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:28:13.496175 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59138e5e-6fcf-4ae5-be6c-214e695e9857 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e79498873d4ed29a0e19bbea9b9214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:28:13.496758 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59138e5e-6fcf-4ae5-be6c-214e695e9857 service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:28:13.497035 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59138e5e-6fcf-4ae5-be6c-214e695e9857 service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:28:13.497306 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59138e5e-6fcf-4ae5-be6c-214e695e9857 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:28:13.505524 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59138e5e-6fcf-4ae5-be6c-214e695e9857 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:28:13.506930 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59138e5e-6fcf-4ae5-be6c-214e695e9857 service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:28:13.515577 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1195/2393] 10.222.0.61 () {60 vars in 1396 bytes} [Mon Aug 22 19:28:13 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3262 bytes in 32 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:28:23.475021 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=13 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:28:23.511604 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=6 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:28:24.697572 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2cee41-46fb-4036-b5f2-c08b4aa0b231 service cinder] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:28:24.709105 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2cee41-46fb-4036-b5f2-c08b4aa0b231 service cinder] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:28:24.714613 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c25768a-bb23-4680-bfb6-a642433267f1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f43e2756b746a6825ca1a06b7d7c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:28:24.715200 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c25768a-bb23-4680-bfb6-a642433267f1 service nova] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:28:24.715498 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c25768a-bb23-4680-bfb6-a642433267f1 service nova] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:28:24.715778 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c25768a-bb23-4680-bfb6-a642433267f1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:28:24.723968 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c25768a-bb23-4680-bfb6-a642433267f1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:28:24.725546 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c25768a-bb23-4680-bfb6-a642433267f1 service nova] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:28:24.753142 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1199/2394] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:28:24 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:28:34.750396 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=13 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:29:28.280355 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59138e5e-6fcf-4ae5-be6c-214e695e9857 service cinder] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:29:28.300246 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59138e5e-6fcf-4ae5-be6c-214e695e9857 service cinder] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:29:28.304014 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6770dd6-f47c-47eb-b3e3-223aed53616d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:29:28.304619 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6770dd6-f47c-47eb-b3e3-223aed53616d service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:29:28.304893 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6770dd6-f47c-47eb-b3e3-223aed53616d service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:29:28.305165 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6770dd6-f47c-47eb-b3e3-223aed53616d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:29:28.313872 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6770dd6-f47c-47eb-b3e3-223aed53616d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:29:28.315305 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6770dd6-f47c-47eb-b3e3-223aed53616d service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:29:28.323920 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1196/2395] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:29:28 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:29:38.320063 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=10 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:29:40.802647 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c25768a-bb23-4680-bfb6-a642433267f1 service nova] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:29:40.812845 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c25768a-bb23-4680-bfb6-a642433267f1 service nova] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:29:40.816455 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dc9e8a-63a1-45f7-baef-1a0110eb4c7c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e79498873d4ed29a0e19bbea9b9214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:29:40.817054 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dc9e8a-63a1-45f7-baef-1a0110eb4c7c service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:29:40.817330 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dc9e8a-63a1-45f7-baef-1a0110eb4c7c service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:29:40.817603 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dc9e8a-63a1-45f7-baef-1a0110eb4c7c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:29:40.828603 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dc9e8a-63a1-45f7-baef-1a0110eb4c7c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:29:40.830022 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dc9e8a-63a1-45f7-baef-1a0110eb4c7c service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:29:40.840832 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1200/2396] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:29:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:29:40.849007 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6770dd6-f47c-47eb-b3e3-223aed53616d service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:29:40.858500 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6770dd6-f47c-47eb-b3e3-223aed53616d service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:29:40.862087 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf0e9ab-eefe-44cd-848b-cebaca7254ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e79498873d4ed29a0e19bbea9b9214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:29:40.862685 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf0e9ab-eefe-44cd-848b-cebaca7254ab service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:29:40.862964 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf0e9ab-eefe-44cd-848b-cebaca7254ab service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:29:40.863232 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf0e9ab-eefe-44cd-848b-cebaca7254ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:29:40.871865 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf0e9ab-eefe-44cd-848b-cebaca7254ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:29:40.873808 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf0e9ab-eefe-44cd-848b-cebaca7254ab service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:29:40.901589 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1197/2397] 10.222.0.61 () {60 vars in 1396 bytes} [Mon Aug 22 19:29:40 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3270 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:29:40.945960 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dc9e8a-63a1-45f7-baef-1a0110eb4c7c service cinder] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:29:40.956066 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dc9e8a-63a1-45f7-baef-1a0110eb4c7c service cinder] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:29:40.960146 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb05cef-be34-49ec-8da2-e1fc0df514c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e79498873d4ed29a0e19bbea9b9214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:29:40.960839 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb05cef-be34-49ec-8da2-e1fc0df514c1 service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:29:40.961190 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb05cef-be34-49ec-8da2-e1fc0df514c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:29:40.961538 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb05cef-be34-49ec-8da2-e1fc0df514c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:29:40.971983 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb05cef-be34-49ec-8da2-e1fc0df514c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:29:40.973665 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb05cef-be34-49ec-8da2-e1fc0df514c1 service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:29:41.011308 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1201/2398] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:29:40 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:29:50.898838 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=13 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:29:51.008501 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=19 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:31:41.316650 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf0e9ab-eefe-44cd-848b-cebaca7254ab service cinder] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:31:41.329279 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf0e9ab-eefe-44cd-848b-cebaca7254ab service cinder] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:31:41.335238 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912dcc33-f284-4ebb-a631-a84f3d61ee9f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f43e2756b746a6825ca1a06b7d7c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:31:41.335898 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912dcc33-f284-4ebb-a631-a84f3d61ee9f service nova] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:31:41.336400 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912dcc33-f284-4ebb-a631-a84f3d61ee9f service nova] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:31:41.336719 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912dcc33-f284-4ebb-a631-a84f3d61ee9f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:31:41.345140 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912dcc33-f284-4ebb-a631-a84f3d61ee9f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:31:41.346711 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912dcc33-f284-4ebb-a631-a84f3d61ee9f service nova] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:31:41.377610 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1198/2399] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:31:41 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:31:51.373614 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=13 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:32:36.523862 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb05cef-be34-49ec-8da2-e1fc0df514c1 service cinder] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:32:36.525319 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-3cb05cef-be34-49ec-8da2-e1fc0df514c1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:32:36.540481 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb05cef-be34-49ec-8da2-e1fc0df514c1 service cinder] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:32:36.545488 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea27e91-7180-4338-a4dc-5294574d19fe service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eef0f73297e04d62bc22acb790fad6ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:32:36.546097 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea27e91-7180-4338-a4dc-5294574d19fe service placement] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:32:36.546373 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea27e91-7180-4338-a4dc-5294574d19fe service placement] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:32:36.546642 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea27e91-7180-4338-a4dc-5294574d19fe service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:32:36.556094 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea27e91-7180-4338-a4dc-5294574d19fe service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:32:36.557674 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea27e91-7180-4338-a4dc-5294574d19fe service placement] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:32:36.572284 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1202/2400] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:32:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:32:46.563515 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=6 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:33:41.511951 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912dcc33-f284-4ebb-a631-a84f3d61ee9f service nova] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:33:41.521723 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912dcc33-f284-4ebb-a631-a84f3d61ee9f service nova] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:33:41.525478 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9a8062-fd6d-47c7-9ff8-53c884a79a8c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e79498873d4ed29a0e19bbea9b9214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:33:41.526084 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9a8062-fd6d-47c7-9ff8-53c884a79a8c service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:33:41.526369 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9a8062-fd6d-47c7-9ff8-53c884a79a8c service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:33:41.528836 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9a8062-fd6d-47c7-9ff8-53c884a79a8c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:33:41.528836 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-0e9a8062-fd6d-47c7-9ff8-53c884a79a8c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:33:41.618585 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9a8062-fd6d-47c7-9ff8-53c884a79a8c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:33:41.620085 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9a8062-fd6d-47c7-9ff8-53c884a79a8c service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:33:41.631833 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1199/2401] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:33:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:33:41.639152 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea27e91-7180-4338-a4dc-5294574d19fe service placement] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:33:41.649580 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea27e91-7180-4338-a4dc-5294574d19fe service placement] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:33:41.653608 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1557278c-e830-4521-b336-29bc2e1d2f38 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e79498873d4ed29a0e19bbea9b9214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:33:41.654218 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1557278c-e830-4521-b336-29bc2e1d2f38 service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:33:41.654218 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1557278c-e830-4521-b336-29bc2e1d2f38 service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:33:41.654218 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1557278c-e830-4521-b336-29bc2e1d2f38 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:33:41.666760 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1557278c-e830-4521-b336-29bc2e1d2f38 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:33:41.671591 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1557278c-e830-4521-b336-29bc2e1d2f38 service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:33:41.697948 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1203/2402] 10.222.0.61 () {60 vars in 1396 bytes} [Mon Aug 22 19:33:41 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3258 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:33:41.893906 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9a8062-fd6d-47c7-9ff8-53c884a79a8c service cinder] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:33:41.895624 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-0e9a8062-fd6d-47c7-9ff8-53c884a79a8c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:33:41.909739 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9a8062-fd6d-47c7-9ff8-53c884a79a8c service cinder] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:33:41.916790 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c765d38d-3c18-40fc-b4cb-1bdbd08bc49d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f43e2756b746a6825ca1a06b7d7c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:33:41.917751 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c765d38d-3c18-40fc-b4cb-1bdbd08bc49d service nova] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:33:41.918033 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c765d38d-3c18-40fc-b4cb-1bdbd08bc49d service nova] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:33:41.918299 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c765d38d-3c18-40fc-b4cb-1bdbd08bc49d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:33:41.920219 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-c765d38d-3c18-40fc-b4cb-1bdbd08bc49d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:33:41.995595 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c765d38d-3c18-40fc-b4cb-1bdbd08bc49d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:33:41.997157 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c765d38d-3c18-40fc-b4cb-1bdbd08bc49d service nova] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:33:42.007545 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1200/2403] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:33:41 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:33:42.064882 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1557278c-e830-4521-b336-29bc2e1d2f38 service cinder] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:33:42.078553 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1557278c-e830-4521-b336-29bc2e1d2f38 service cinder] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:33:42.089464 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ab4df2-f5bc-4368-b2b4-237343742cad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f43e2756b746a6825ca1a06b7d7c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:33:42.090094 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ab4df2-f5bc-4368-b2b4-237343742cad service nova] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:33:42.090368 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ab4df2-f5bc-4368-b2b4-237343742cad service nova] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:33:42.090635 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ab4df2-f5bc-4368-b2b4-237343742cad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:33:42.092931 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-c1ab4df2-f5bc-4368-b2b4-237343742cad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:33:42.179133 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ab4df2-f5bc-4368-b2b4-237343742cad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:33:42.180755 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ab4df2-f5bc-4368-b2b4-237343742cad service nova] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:33:42.199626 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1204/2404] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:33:42 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:33:52.002865 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=72 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:33:52.197086 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=54 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:34:31.173894 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c765d38d-3c18-40fc-b4cb-1bdbd08bc49d service nova] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:34:31.175569 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-c765d38d-3c18-40fc-b4cb-1bdbd08bc49d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:34:31.245147 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c765d38d-3c18-40fc-b4cb-1bdbd08bc49d service nova] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:34:31.248846 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd20d961-df41-4641-8feb-57cbb8f125a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:34:31.249452 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd20d961-df41-4641-8feb-57cbb8f125a0 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:34:31.249731 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd20d961-df41-4641-8feb-57cbb8f125a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:34:31.250011 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd20d961-df41-4641-8feb-57cbb8f125a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:34:31.252519 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-dd20d961-df41-4641-8feb-57cbb8f125a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:34:31.262268 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd20d961-df41-4641-8feb-57cbb8f125a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:34:31.263774 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd20d961-df41-4641-8feb-57cbb8f125a0 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:34:31.271837 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1201/2405] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:34:31 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:34:41.268351 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=31 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:35:41.870137 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ab4df2-f5bc-4368-b2b4-237343742cad service nova] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:35:41.879435 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ab4df2-f5bc-4368-b2b4-237343742cad service nova] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:35:41.883829 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c408b0-a54f-4c2f-910f-852c1fb818ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e79498873d4ed29a0e19bbea9b9214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:35:41.884437 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c408b0-a54f-4c2f-910f-852c1fb818ba service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:35:41.884716 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c408b0-a54f-4c2f-910f-852c1fb818ba service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:35:41.884991 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c408b0-a54f-4c2f-910f-852c1fb818ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:35:41.887250 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-54c408b0-a54f-4c2f-910f-852c1fb818ba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:35:41.901610 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c408b0-a54f-4c2f-910f-852c1fb818ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:35:41.903038 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c408b0-a54f-4c2f-910f-852c1fb818ba service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:35:41.914124 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1205/2406] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:35:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:35:41.921993 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd20d961-df41-4641-8feb-57cbb8f125a0 service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:35:41.931424 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd20d961-df41-4641-8feb-57cbb8f125a0 service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:35:41.935755 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7409eb41-b774-461f-ac7c-f6e4dc992cb0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e79498873d4ed29a0e19bbea9b9214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:35:41.936387 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7409eb41-b774-461f-ac7c-f6e4dc992cb0 service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:35:41.936666 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7409eb41-b774-461f-ac7c-f6e4dc992cb0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:35:41.936938 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7409eb41-b774-461f-ac7c-f6e4dc992cb0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:35:41.938833 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-7409eb41-b774-461f-ac7c-f6e4dc992cb0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:35:42.010524 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7409eb41-b774-461f-ac7c-f6e4dc992cb0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:35:42.011965 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7409eb41-b774-461f-ac7c-f6e4dc992cb0 service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:35:42.020826 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1202/2407] 10.222.0.61 () {60 vars in 1396 bytes} [Mon Aug 22 19:35:41 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3270 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:35:42.179044 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c408b0-a54f-4c2f-910f-852c1fb818ba service cinder] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:35:42.191969 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c408b0-a54f-4c2f-910f-852c1fb818ba service cinder] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:35:42.199052 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f98a61f-6b8d-499e-8e7b-f774d90e7240 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f43e2756b746a6825ca1a06b7d7c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:35:42.199052 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f98a61f-6b8d-499e-8e7b-f774d90e7240 service nova] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:35:42.199052 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f98a61f-6b8d-499e-8e7b-f774d90e7240 service nova] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:35:42.199052 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f98a61f-6b8d-499e-8e7b-f774d90e7240 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:35:42.205178 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-8f98a61f-6b8d-499e-8e7b-f774d90e7240 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:35:42.290783 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f98a61f-6b8d-499e-8e7b-f774d90e7240 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:35:42.292553 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f98a61f-6b8d-499e-8e7b-f774d90e7240 service nova] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:35:42.302793 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1206/2408] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:35:42 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:35:52.016799 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=33 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:35:52.298324 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=39 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:36:40.917605 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7409eb41-b774-461f-ac7c-f6e4dc992cb0 service cinder] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:36:40.926330 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7409eb41-b774-461f-ac7c-f6e4dc992cb0 service cinder] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:36:40.930562 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbb710b-fa07-4dc9-ae6d-d0743af58ef6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e79498873d4ed29a0e19bbea9b9214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:36:40.931167 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbb710b-fa07-4dc9-ae6d-d0743af58ef6 service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:36:40.931439 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbb710b-fa07-4dc9-ae6d-d0743af58ef6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:36:40.931734 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbb710b-fa07-4dc9-ae6d-d0743af58ef6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:36:40.933559 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-dfbb710b-fa07-4dc9-ae6d-d0743af58ef6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:36:41.005783 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbb710b-fa07-4dc9-ae6d-d0743af58ef6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:36:41.007790 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbb710b-fa07-4dc9-ae6d-d0743af58ef6 service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:36:41.017044 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1203/2409] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:36:40 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:36:51.012647 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=33 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:37:40.527313 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f98a61f-6b8d-499e-8e7b-f774d90e7240 service nova] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:37:40.668427 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f98a61f-6b8d-499e-8e7b-f774d90e7240 service nova] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:37:40.673751 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbb094b-a195-4c71-a719-d07eda591c65 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eef0f73297e04d62bc22acb790fad6ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:37:40.674379 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbb094b-a195-4c71-a719-d07eda591c65 service placement] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:37:40.674673 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbb094b-a195-4c71-a719-d07eda591c65 service placement] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:37:40.674934 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbb094b-a195-4c71-a719-d07eda591c65 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:37:40.676888 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-0bbb094b-a195-4c71-a719-d07eda591c65 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:37:40.713958 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbb094b-a195-4c71-a719-d07eda591c65 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:37:40.715564 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbb094b-a195-4c71-a719-d07eda591c65 service placement] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:37:40.725893 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1207/2410] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:37:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:37:50.721577 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=63 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:38:41.956175 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbb710b-fa07-4dc9-ae6d-d0743af58ef6 service cinder] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:38:41.957816 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-dfbb710b-fa07-4dc9-ae6d-d0743af58ef6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:38:42.053355 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbb710b-fa07-4dc9-ae6d-d0743af58ef6 service cinder] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:38:42.056845 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b85fce2-12b9-4200-9660-9f6fd04f92ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e79498873d4ed29a0e19bbea9b9214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:38:42.057457 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b85fce2-12b9-4200-9660-9f6fd04f92ca service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:38:42.057714 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b85fce2-12b9-4200-9660-9f6fd04f92ca service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:38:42.058009 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b85fce2-12b9-4200-9660-9f6fd04f92ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:38:42.059844 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-7b85fce2-12b9-4200-9660-9f6fd04f92ca service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:38:42.130128 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b85fce2-12b9-4200-9660-9f6fd04f92ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:38:42.131548 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b85fce2-12b9-4200-9660-9f6fd04f92ca service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:38:42.140308 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1204/2411] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:38:41 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:38:42.187737 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbb094b-a195-4c71-a719-d07eda591c65 service placement] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:38:42.196924 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbb094b-a195-4c71-a719-d07eda591c65 service placement] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:38:42.200513 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805d5398-4a11-48ff-91b1-2518d8f114eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e79498873d4ed29a0e19bbea9b9214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:38:42.201117 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805d5398-4a11-48ff-91b1-2518d8f114eb service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:38:42.201396 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805d5398-4a11-48ff-91b1-2518d8f114eb service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:38:42.201670 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805d5398-4a11-48ff-91b1-2518d8f114eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:38:42.210252 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805d5398-4a11-48ff-91b1-2518d8f114eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:38:42.211692 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805d5398-4a11-48ff-91b1-2518d8f114eb service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:38:42.239660 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1208/2412] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:38:42 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:38:43.342466 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b85fce2-12b9-4200-9660-9f6fd04f92ca service cinder] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:38:43.351435 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b85fce2-12b9-4200-9660-9f6fd04f92ca service cinder] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:38:43.354970 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd535e1c-b04a-417a-b7b0-d728e7da222e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e79498873d4ed29a0e19bbea9b9214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:38:43.355555 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd535e1c-b04a-417a-b7b0-d728e7da222e service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:38:43.355860 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd535e1c-b04a-417a-b7b0-d728e7da222e service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:38:43.356128 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd535e1c-b04a-417a-b7b0-d728e7da222e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:38:43.364228 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd535e1c-b04a-417a-b7b0-d728e7da222e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:38:43.365622 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd535e1c-b04a-417a-b7b0-d728e7da222e service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:38:43.374250 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1205/2413] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:38:43 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 32 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:38:52.237094 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=13 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:38:53.370284 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=77 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:39:36.196675 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805d5398-4a11-48ff-91b1-2518d8f114eb service cinder] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:39:36.216478 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805d5398-4a11-48ff-91b1-2518d8f114eb service cinder] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:39:36.220254 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c79a80-5224-46ad-8d07-fc192bff9386 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:39:36.220884 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c79a80-5224-46ad-8d07-fc192bff9386 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:39:36.221188 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c79a80-5224-46ad-8d07-fc192bff9386 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:39:36.221454 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c79a80-5224-46ad-8d07-fc192bff9386 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:39:36.228752 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c79a80-5224-46ad-8d07-fc192bff9386 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:39:36.230100 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c79a80-5224-46ad-8d07-fc192bff9386 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:39:36.237898 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1209/2414] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:39:36 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:39:46.234420 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=10 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:40:41.441314 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd535e1c-b04a-417a-b7b0-d728e7da222e service cinder] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:40:41.442908 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-cd535e1c-b04a-417a-b7b0-d728e7da222e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:40:41.454920 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd535e1c-b04a-417a-b7b0-d728e7da222e service cinder] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:40:41.459023 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f8b450-36ea-4c6e-9708-7124858cd5ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e79498873d4ed29a0e19bbea9b9214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:40:41.459620 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f8b450-36ea-4c6e-9708-7124858cd5ad service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:40:41.459947 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f8b450-36ea-4c6e-9708-7124858cd5ad service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:40:41.460220 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f8b450-36ea-4c6e-9708-7124858cd5ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:40:41.469319 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f8b450-36ea-4c6e-9708-7124858cd5ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:40:41.470747 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f8b450-36ea-4c6e-9708-7124858cd5ad service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:40:41.499588 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1206/2415] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:40:41 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:40:41.766417 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c79a80-5224-46ad-8d07-fc192bff9386 service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:40:41.796278 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c79a80-5224-46ad-8d07-fc192bff9386 service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:40:41.801695 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601fc917-3cfa-4fbf-9056-2d59dcc60902 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f43e2756b746a6825ca1a06b7d7c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:40:41.802310 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601fc917-3cfa-4fbf-9056-2d59dcc60902 service nova] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:40:41.802584 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601fc917-3cfa-4fbf-9056-2d59dcc60902 service nova] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:40:41.802850 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601fc917-3cfa-4fbf-9056-2d59dcc60902 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:40:41.811193 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601fc917-3cfa-4fbf-9056-2d59dcc60902 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:40:41.812786 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601fc917-3cfa-4fbf-9056-2d59dcc60902 service nova] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:40:41.840128 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1210/2416] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:40:41 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:40:51.496472 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=13 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:40:51.837525 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=20 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:41:41.693079 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f8b450-36ea-4c6e-9708-7124858cd5ad service cinder] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:41:41.702583 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f8b450-36ea-4c6e-9708-7124858cd5ad service cinder] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:41:41.706209 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7b4fdc-c753-460b-b50d-80d8f7a29d62 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e79498873d4ed29a0e19bbea9b9214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:41:41.707057 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7b4fdc-c753-460b-b50d-80d8f7a29d62 service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:41:41.707333 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7b4fdc-c753-460b-b50d-80d8f7a29d62 service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:41:41.707604 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7b4fdc-c753-460b-b50d-80d8f7a29d62 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:41:41.715913 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7b4fdc-c753-460b-b50d-80d8f7a29d62 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:41:41.717319 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7b4fdc-c753-460b-b50d-80d8f7a29d62 service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:41:41.743881 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1207/2417] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:41:41 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:41:51.741243 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=13 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:42:14.978287 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601fc917-3cfa-4fbf-9056-2d59dcc60902 service nova] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:42:14.988388 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601fc917-3cfa-4fbf-9056-2d59dcc60902 service nova] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:42:14.992073 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28456f4f-4090-45f0-a196-fbd2475daf80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:42:14.992670 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28456f4f-4090-45f0-a196-fbd2475daf80 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:42:14.992948 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28456f4f-4090-45f0-a196-fbd2475daf80 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:42:14.993223 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28456f4f-4090-45f0-a196-fbd2475daf80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:42:15.003958 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28456f4f-4090-45f0-a196-fbd2475daf80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:42:15.005375 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28456f4f-4090-45f0-a196-fbd2475daf80 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:42:15.016302 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1211/2418] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:42:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:42:15.970879 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7b4fdc-c753-460b-b50d-80d8f7a29d62 service cinder] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:42:15.972579 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-5c7b4fdc-c753-460b-b50d-80d8f7a29d62 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:42:15.984929 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7b4fdc-c753-460b-b50d-80d8f7a29d62 service cinder] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:42:15.988590 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b0de60-b14e-49e0-b36c-8bf287966e52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:42:15.989196 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b0de60-b14e-49e0-b36c-8bf287966e52 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:42:15.989475 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b0de60-b14e-49e0-b36c-8bf287966e52 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:42:15.989748 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b0de60-b14e-49e0-b36c-8bf287966e52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:42:15.991509 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-81b0de60-b14e-49e0-b36c-8bf287966e52 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:42:16.075928 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b0de60-b14e-49e0-b36c-8bf287966e52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:42:16.077331 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b0de60-b14e-49e0-b36c-8bf287966e52 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:42:16.096896 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1208/2419] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:42:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:42:17.158104 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28456f4f-4090-45f0-a196-fbd2475daf80 service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:42:17.172058 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28456f4f-4090-45f0-a196-fbd2475daf80 service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:42:17.176388 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e7b171-7dcd-4680-a076-6b310c5070de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:42:17.177009 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e7b171-7dcd-4680-a076-6b310c5070de service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:42:17.177368 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e7b171-7dcd-4680-a076-6b310c5070de service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:42:17.177684 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e7b171-7dcd-4680-a076-6b310c5070de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:42:17.190349 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e7b171-7dcd-4680-a076-6b310c5070de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:42:17.191986 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e7b171-7dcd-4680-a076-6b310c5070de service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:42:17.206112 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1212/2420] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:42:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:42:19.568476 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ce9c51-9614-497b-87b3-50c7ade4b53a None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:42:19.569044 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ce9c51-9614-497b-87b3-50c7ade4b53a None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:42:19.569044 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ce9c51-9614-497b-87b3-50c7ade4b53a None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:42:19.570424 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1209/2421] 10.222.0.61 () {54 vars in 921 bytes} [Mon Aug 22 19:42:19 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 19:42:19.577500 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24706780-8da1-4adc-97b2-c1d1144ff347 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:42:19.577973 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24706780-8da1-4adc-97b2-c1d1144ff347 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:42:19.578134 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24706780-8da1-4adc-97b2-c1d1144ff347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:42:19.643404 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-24706780-8da1-4adc-97b2-c1d1144ff347 None None] MFA Rules not processed for user `eef0f73297e04d62bc22acb790fad6ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:42:19.645072 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-24706780-8da1-4adc-97b2-c1d1144ff347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:42:19.658276 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1213/2422] 10.222.0.61 () {58 vars in 1018 bytes} [Mon Aug 22 19:42:19 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:42:19.674498 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ce9c51-9614-497b-87b3-50c7ade4b53a None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:42:19.676445 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-f5ce9c51-9614-497b-87b3-50c7ade4b53a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:42:19.701173 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ce9c51-9614-497b-87b3-50c7ade4b53a None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:42:19.708159 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62293f22-49ff-4916-8a41-b2cce24bbd57 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eef0f73297e04d62bc22acb790fad6ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:42:19.708769 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62293f22-49ff-4916-8a41-b2cce24bbd57 service placement] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:42:19.709045 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62293f22-49ff-4916-8a41-b2cce24bbd57 service placement] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:42:19.709314 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62293f22-49ff-4916-8a41-b2cce24bbd57 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:42:19.721678 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62293f22-49ff-4916-8a41-b2cce24bbd57 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:42:19.724981 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62293f22-49ff-4916-8a41-b2cce24bbd57 service placement] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:42:19.740941 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1210/2423] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:42:19 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:42:23.123148 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24706780-8da1-4adc-97b2-c1d1144ff347 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:42:23.124907 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-24706780-8da1-4adc-97b2-c1d1144ff347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:42:23.139996 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24706780-8da1-4adc-97b2-c1d1144ff347 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:42:23.145906 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df2462d-e257-4dad-9e26-edcc92d13997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:42:23.146547 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df2462d-e257-4dad-9e26-edcc92d13997 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:42:23.146890 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df2462d-e257-4dad-9e26-edcc92d13997 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:42:23.147173 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df2462d-e257-4dad-9e26-edcc92d13997 admin admin] PATH_INFO: `/v3/users/a1b3705aa22743c49e3ac2f37c7f2084` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:42:23.148308 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df2462d-e257-4dad-9e26-edcc92d13997 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1b3705aa22743c49e3ac2f37c7f2084)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:42:23.150288 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df2462d-e257-4dad-9e26-edcc92d13997 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:42:23.200557 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-4df2462d-e257-4dad-9e26-edcc92d13997 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1b3705aa22743c49e3ac2f37c7f2084'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:42:23.201302 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-4df2462d-e257-4dad-9e26-edcc92d13997 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1b3705aa22743c49e3ac2f37c7f2084'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:42:23.209172 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-4df2462d-e257-4dad-9e26-edcc92d13997 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1b3705aa22743c49e3ac2f37c7f2084'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:42:23.216759 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-4df2462d-e257-4dad-9e26-edcc92d13997 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1b3705aa22743c49e3ac2f37c7f2084'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:42:23.230348 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-4df2462d-e257-4dad-9e26-edcc92d13997 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1b3705aa22743c49e3ac2f37c7f2084'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:42:23.236468 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1214/2424] 10.222.0.61 () {58 vars in 1224 bytes} [Mon Aug 22 19:42:23 2022] DELETE /identity/v3/users/a1b3705aa22743c49e3ac2f37c7f2084 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:42:23.416001 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62293f22-49ff-4916-8a41-b2cce24bbd57 service placement] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:42:23.427573 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62293f22-49ff-4916-8a41-b2cce24bbd57 service placement] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:42:23.495090 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09453bbd-47b8-4844-acbe-ddfb89d1635e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:42:23.495712 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09453bbd-47b8-4844-acbe-ddfb89d1635e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:42:23.495992 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09453bbd-47b8-4844-acbe-ddfb89d1635e admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:42:23.496259 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09453bbd-47b8-4844-acbe-ddfb89d1635e admin admin] PATH_INFO: `/v3/projects/e3adbe4feef048239c2020ac458e11b0` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:42:23.497459 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09453bbd-47b8-4844-acbe-ddfb89d1635e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3adbe4feef048239c2020ac458e11b0)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:42:23.499475 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09453bbd-47b8-4844-acbe-ddfb89d1635e admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:42:23.549184 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-09453bbd-47b8-4844-acbe-ddfb89d1635e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3adbe4feef048239c2020ac458e11b0', 'request_id': 'req-09453bbd-47b8-4844-acbe-ddfb89d1635e'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:42:23.552818 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:42:23.558577 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-09453bbd-47b8-4844-acbe-ddfb89d1635e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3adbe4feef048239c2020ac458e11b0', 'request_id': 'req-09453bbd-47b8-4844-acbe-ddfb89d1635e'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:42:23.567328 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-09453bbd-47b8-4844-acbe-ddfb89d1635e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3adbe4feef048239c2020ac458e11b0', 'request_id': 'req-09453bbd-47b8-4844-acbe-ddfb89d1635e'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:42:23.581162 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-09453bbd-47b8-4844-acbe-ddfb89d1635e admin admin] The token cache is being invalidate because project e3adbe4feef048239c2020ac458e11b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 19:42:23.581729 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-09453bbd-47b8-4844-acbe-ddfb89d1635e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:42:23.582648 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-09453bbd-47b8-4844-acbe-ddfb89d1635e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:42:23.584288 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1211/2425] 10.222.0.61 () {58 vars in 1233 bytes} [Mon Aug 22 19:42:23 2022] DELETE /identity/v3/projects/e3adbe4feef048239c2020ac458e11b0 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:42:23.588447 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df2462d-e257-4dad-9e26-edcc92d13997 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:42:23.590018 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-4df2462d-e257-4dad-9e26-edcc92d13997 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:42:23.603245 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df2462d-e257-4dad-9e26-edcc92d13997 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:42:23.668892 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc42e156-63ee-414d-b830-0f7f2d74aef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:42:23.669944 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc42e156-63ee-414d-b830-0f7f2d74aef2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:42:23.670089 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc42e156-63ee-414d-b830-0f7f2d74aef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:42:23.670281 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc42e156-63ee-414d-b830-0f7f2d74aef2 admin admin] PATH_INFO: `/v3/users/d4deb36470284a368a6bf4fde96c6a33` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:42:23.684408 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:42:23.687867 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc42e156-63ee-414d-b830-0f7f2d74aef2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4deb36470284a368a6bf4fde96c6a33)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:42:23.690869 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc42e156-63ee-414d-b830-0f7f2d74aef2 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:42:23.752990 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-dc42e156-63ee-414d-b830-0f7f2d74aef2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4deb36470284a368a6bf4fde96c6a33'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:42:23.753646 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-dc42e156-63ee-414d-b830-0f7f2d74aef2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4deb36470284a368a6bf4fde96c6a33'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:42:23.765046 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-dc42e156-63ee-414d-b830-0f7f2d74aef2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4deb36470284a368a6bf4fde96c6a33'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:42:23.775389 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-dc42e156-63ee-414d-b830-0f7f2d74aef2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4deb36470284a368a6bf4fde96c6a33'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:42:23.796707 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-dc42e156-63ee-414d-b830-0f7f2d74aef2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4deb36470284a368a6bf4fde96c6a33'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:42:23.803984 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1215/2426] 10.222.0.61 () {58 vars in 1224 bytes} [Mon Aug 22 19:42:23 2022] DELETE /identity/v3/users/d4deb36470284a368a6bf4fde96c6a33 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:42:23.973308 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09453bbd-47b8-4844-acbe-ddfb89d1635e admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:42:23.974825 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-09453bbd-47b8-4844-acbe-ddfb89d1635e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:42:23.992209 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09453bbd-47b8-4844-acbe-ddfb89d1635e admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:42:24.048053 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25313b9c-e920-4e3f-a0a7-0d755a6d085a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:42:24.048685 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25313b9c-e920-4e3f-a0a7-0d755a6d085a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:42:24.049485 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25313b9c-e920-4e3f-a0a7-0d755a6d085a admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:42:24.049748 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25313b9c-e920-4e3f-a0a7-0d755a6d085a admin admin] PATH_INFO: `/v3/projects/97b0943c57664ba6acbdc01928fc178e` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:42:24.059889 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25313b9c-e920-4e3f-a0a7-0d755a6d085a admin admin] RBAC: Authorizing `identity:delete_project(project_id=97b0943c57664ba6acbdc01928fc178e)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:42:24.061789 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25313b9c-e920-4e3f-a0a7-0d755a6d085a admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:42:24.107386 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-25313b9c-e920-4e3f-a0a7-0d755a6d085a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97b0943c57664ba6acbdc01928fc178e', 'request_id': 'req-25313b9c-e920-4e3f-a0a7-0d755a6d085a'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:42:24.115818 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-25313b9c-e920-4e3f-a0a7-0d755a6d085a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97b0943c57664ba6acbdc01928fc178e', 'request_id': 'req-25313b9c-e920-4e3f-a0a7-0d755a6d085a'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:42:24.122854 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-25313b9c-e920-4e3f-a0a7-0d755a6d085a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97b0943c57664ba6acbdc01928fc178e', 'request_id': 'req-25313b9c-e920-4e3f-a0a7-0d755a6d085a'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:42:24.135322 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-25313b9c-e920-4e3f-a0a7-0d755a6d085a admin admin] The token cache is being invalidate because project 97b0943c57664ba6acbdc01928fc178e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 19:42:24.135614 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-25313b9c-e920-4e3f-a0a7-0d755a6d085a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:42:24.136296 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-25313b9c-e920-4e3f-a0a7-0d755a6d085a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:42:24.137970 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1212/2427] 10.222.0.61 () {58 vars in 1233 bytes} [Mon Aug 22 19:42:23 2022] DELETE /identity/v3/projects/97b0943c57664ba6acbdc01928fc178e => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:42:33.800355 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=27,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:42:34.130916 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=61,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:42:44.499416 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc42e156-63ee-414d-b830-0f7f2d74aef2 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:42:44.500969 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-dc42e156-63ee-414d-b830-0f7f2d74aef2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:42:44.573184 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc42e156-63ee-414d-b830-0f7f2d74aef2 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:42:44.641236 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d89765-b9e2-4df1-a6a5-928d1db03c91 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eef0f73297e04d62bc22acb790fad6ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:42:44.642284 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d89765-b9e2-4df1-a6a5-928d1db03c91 service placement] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:42:44.642773 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d89765-b9e2-4df1-a6a5-928d1db03c91 service placement] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:42:44.643264 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d89765-b9e2-4df1-a6a5-928d1db03c91 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:42:44.646748 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-84d89765-b9e2-4df1-a6a5-928d1db03c91 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:42:44.664679 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d89765-b9e2-4df1-a6a5-928d1db03c91 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:42:44.666289 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d89765-b9e2-4df1-a6a5-928d1db03c91 service placement] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:42:44.676385 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1216/2428] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:42:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:42:54.672030 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=58 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:43:42.173026 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25313b9c-e920-4e3f-a0a7-0d755a6d085a admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:43:42.174522 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-25313b9c-e920-4e3f-a0a7-0d755a6d085a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:43:42.258982 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25313b9c-e920-4e3f-a0a7-0d755a6d085a admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:43:42.264394 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c41bed-12e4-4640-8b76-96b18f9835a3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f43e2756b746a6825ca1a06b7d7c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:43:42.264996 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c41bed-12e4-4640-8b76-96b18f9835a3 service nova] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:43:42.265274 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c41bed-12e4-4640-8b76-96b18f9835a3 service nova] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:43:42.265545 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c41bed-12e4-4640-8b76-96b18f9835a3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:43:42.267265 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-e5c41bed-12e4-4640-8b76-96b18f9835a3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:43:42.299972 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d89765-b9e2-4df1-a6a5-928d1db03c91 service placement] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:43:42.301706 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-84d89765-b9e2-4df1-a6a5-928d1db03c91 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:43:42.316996 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d89765-b9e2-4df1-a6a5-928d1db03c91 service placement] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:43:42.322600 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e5117b-b09b-47c1-b337-108f8d91e3af service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f43e2756b746a6825ca1a06b7d7c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:43:42.323197 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e5117b-b09b-47c1-b337-108f8d91e3af service nova] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:43:42.323472 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e5117b-b09b-47c1-b337-108f8d91e3af service nova] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:43:42.323770 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e5117b-b09b-47c1-b337-108f8d91e3af service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:43:42.325444 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-c4e5117b-b09b-47c1-b337-108f8d91e3af service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:43:42.337674 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c41bed-12e4-4640-8b76-96b18f9835a3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:43:42.340125 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c41bed-12e4-4640-8b76-96b18f9835a3 service nova] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:43:42.350060 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1213/2429] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:43:42 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:43:42.409355 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e5117b-b09b-47c1-b337-108f8d91e3af service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:43:42.410886 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e5117b-b09b-47c1-b337-108f8d91e3af service nova] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:43:42.430296 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1217/2430] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:43:42 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 19:43:42.938508 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c41bed-12e4-4640-8b76-96b18f9835a3 service nova] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:43:42.950221 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c41bed-12e4-4640-8b76-96b18f9835a3 service nova] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:43:42.955753 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd12570-ca1c-474a-930e-66fc06364bd4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f43e2756b746a6825ca1a06b7d7c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:43:42.956343 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd12570-ca1c-474a-930e-66fc06364bd4 service nova] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:43:42.956618 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd12570-ca1c-474a-930e-66fc06364bd4 service nova] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:43:42.956881 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd12570-ca1c-474a-930e-66fc06364bd4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:43:42.958860 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-6cd12570-ca1c-474a-930e-66fc06364bd4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:43:43.035063 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:43:43.039255 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd12570-ca1c-474a-930e-66fc06364bd4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:43:43.041089 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd12570-ca1c-474a-930e-66fc06364bd4 service nova] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:43:43.059140 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1214/2431] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:43:42 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:43:52.427632 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=36 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:43:53.056511 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=7 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:44:16.448231 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e5117b-b09b-47c1-b337-108f8d91e3af service nova] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:44:16.463551 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e5117b-b09b-47c1-b337-108f8d91e3af service nova] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:44:16.470623 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2fce14-795f-4f79-b170-b60b147ee8b6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f43e2756b746a6825ca1a06b7d7c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:44:16.471470 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2fce14-795f-4f79-b170-b60b147ee8b6 service nova] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:44:16.471801 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2fce14-795f-4f79-b170-b60b147ee8b6 service nova] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:44:16.472074 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2fce14-795f-4f79-b170-b60b147ee8b6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:44:16.474028 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-3f2fce14-795f-4f79-b170-b60b147ee8b6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:44:16.491941 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2fce14-795f-4f79-b170-b60b147ee8b6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:44:16.496195 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2fce14-795f-4f79-b170-b60b147ee8b6 service nova] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:44:16.514089 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1218/2432] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:44:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:44:26.506290 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=6 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:44:41.196347 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd12570-ca1c-474a-930e-66fc06364bd4 service nova] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:44:41.197893 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-6cd12570-ca1c-474a-930e-66fc06364bd4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:44:41.266061 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd12570-ca1c-474a-930e-66fc06364bd4 service nova] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:44:41.270004 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff897b1-9833-4adf-bd06-0661a960f9b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:44:41.270590 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff897b1-9833-4adf-bd06-0661a960f9b4 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:44:41.270869 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff897b1-9833-4adf-bd06-0661a960f9b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:44:41.271138 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff897b1-9833-4adf-bd06-0661a960f9b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:44:41.273155 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-8ff897b1-9833-4adf-bd06-0661a960f9b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:44:41.283538 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff897b1-9833-4adf-bd06-0661a960f9b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:44:41.285016 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff897b1-9833-4adf-bd06-0661a960f9b4 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:44:41.293239 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1215/2433] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:44:41 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:44:51.289579 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=31 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:45:41.875878 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2fce14-795f-4f79-b170-b60b147ee8b6 service nova] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:45:41.887560 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2fce14-795f-4f79-b170-b60b147ee8b6 service nova] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:45:41.893225 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263b1deb-5682-4f51-b3e1-7331a0a49015 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f43e2756b746a6825ca1a06b7d7c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:45:41.893833 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263b1deb-5682-4f51-b3e1-7331a0a49015 service nova] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:45:41.894111 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263b1deb-5682-4f51-b3e1-7331a0a49015 service nova] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:45:41.894409 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263b1deb-5682-4f51-b3e1-7331a0a49015 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:45:41.896166 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-263b1deb-5682-4f51-b3e1-7331a0a49015 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:45:41.954965 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263b1deb-5682-4f51-b3e1-7331a0a49015 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:45:41.956570 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263b1deb-5682-4f51-b3e1-7331a0a49015 service nova] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:45:41.966215 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1219/2434] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:45:41 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:45:42.833563 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff897b1-9833-4adf-bd06-0661a960f9b4 service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:45:42.835579 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-8ff897b1-9833-4adf-bd06-0661a960f9b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:45:42.895675 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff897b1-9833-4adf-bd06-0661a960f9b4 service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:45:42.899191 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2374ea59-00c7-4879-abb6-00ae6ce26bff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e79498873d4ed29a0e19bbea9b9214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:45:42.899821 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2374ea59-00c7-4879-abb6-00ae6ce26bff service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:45:42.900205 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2374ea59-00c7-4879-abb6-00ae6ce26bff service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:45:42.900478 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2374ea59-00c7-4879-abb6-00ae6ce26bff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:45:42.902262 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-2374ea59-00c7-4879-abb6-00ae6ce26bff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:45:42.977061 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2374ea59-00c7-4879-abb6-00ae6ce26bff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:45:42.978485 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2374ea59-00c7-4879-abb6-00ae6ce26bff service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:45:42.990689 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1216/2435] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:45:42 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:45:51.962159 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=28 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:45:52.983523 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=53 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:47:14.269241 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263b1deb-5682-4f51-b3e1-7331a0a49015 service nova] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:47:14.278660 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263b1deb-5682-4f51-b3e1-7331a0a49015 service nova] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:47:14.283325 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691aa949-667f-4953-a1a0-e03a701d0b26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:47:14.283976 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691aa949-667f-4953-a1a0-e03a701d0b26 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:47:14.284253 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691aa949-667f-4953-a1a0-e03a701d0b26 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:47:14.284811 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691aa949-667f-4953-a1a0-e03a701d0b26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:47:14.286553 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-691aa949-667f-4953-a1a0-e03a701d0b26 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:47:14.300768 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691aa949-667f-4953-a1a0-e03a701d0b26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:47:14.302188 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691aa949-667f-4953-a1a0-e03a701d0b26 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:47:14.313507 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1220/2436] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:47:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:47:17.754423 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2374ea59-00c7-4879-abb6-00ae6ce26bff service cinder] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:47:17.755977 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-2374ea59-00c7-4879-abb6-00ae6ce26bff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:47:17.767454 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2374ea59-00c7-4879-abb6-00ae6ce26bff service cinder] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:47:17.771010 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e756e7ff-05c0-453b-83d2-5f268278f89b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:47:17.771597 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e756e7ff-05c0-453b-83d2-5f268278f89b service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:47:17.771897 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e756e7ff-05c0-453b-83d2-5f268278f89b service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:47:17.772164 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e756e7ff-05c0-453b-83d2-5f268278f89b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:47:17.773814 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-e756e7ff-05c0-453b-83d2-5f268278f89b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:47:17.787145 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e756e7ff-05c0-453b-83d2-5f268278f89b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:47:17.788597 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e756e7ff-05c0-453b-83d2-5f268278f89b service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:47:17.799795 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1217/2437] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:47:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:47:24.308300 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=6 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:47:27.794456 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=6 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:47:46.522221 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691aa949-667f-4953-a1a0-e03a701d0b26 service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:47:46.523780 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-691aa949-667f-4953-a1a0-e03a701d0b26 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:47:46.588757 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691aa949-667f-4953-a1a0-e03a701d0b26 service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:47:46.593978 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058e5ed7-0920-4597-9116-eafd4776efa9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eef0f73297e04d62bc22acb790fad6ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:47:46.594575 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058e5ed7-0920-4597-9116-eafd4776efa9 service placement] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:47:46.594852 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058e5ed7-0920-4597-9116-eafd4776efa9 service placement] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:47:46.595120 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058e5ed7-0920-4597-9116-eafd4776efa9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:47:46.603914 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058e5ed7-0920-4597-9116-eafd4776efa9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:47:46.605508 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058e5ed7-0920-4597-9116-eafd4776efa9 service placement] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:47:46.615583 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1221/2438] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:47:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:47:56.610903 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=28 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:48:17.947772 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce130b3-e496-4b03-a8fe-c7b039a8f229 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:48:17.948316 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce130b3-e496-4b03-a8fe-c7b039a8f229 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:48:17.948316 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce130b3-e496-4b03-a8fe-c7b039a8f229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:48:17.999593 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-9ce130b3-e496-4b03-a8fe-c7b039a8f229 None None] MFA Rules not processed for user `399396a8fe254da199bc48b0ca5243d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:48:18.000889 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-9ce130b3-e496-4b03-a8fe-c7b039a8f229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:48:18.072118 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1218/2439] 10.222.0.61 () {58 vars in 967 bytes} [Mon Aug 22 19:48:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:48:18.118848 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058e5ed7-0920-4597-9116-eafd4776efa9 service placement] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:48:18.120458 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-058e5ed7-0920-4597-9116-eafd4776efa9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:48:18.255751 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058e5ed7-0920-4597-9116-eafd4776efa9 service placement] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:48:18.265002 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf0ce94-8d75-4532-96fd-7f66b45f3176 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2824359491934d0dacaa21a718becee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:48:18.266551 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf0ce94-8d75-4532-96fd-7f66b45f3176 service swift] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:48:18.267011 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf0ce94-8d75-4532-96fd-7f66b45f3176 service swift] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:48:18.267453 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf0ce94-8d75-4532-96fd-7f66b45f3176 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:48:18.281271 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf0ce94-8d75-4532-96fd-7f66b45f3176 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:48:18.284155 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf0ce94-8d75-4532-96fd-7f66b45f3176 service swift] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:48:18.296989 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1222/2440] 10.222.0.61 () {60 vars in 1384 bytes} [Mon Aug 22 19:48:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 22 19:48:18.885700 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce130b3-e496-4b03-a8fe-c7b039a8f229 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:48:18.895550 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce130b3-e496-4b03-a8fe-c7b039a8f229 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:48:18.900229 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df3b052-e0d8-4f30-9c58-2fb9473403fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:48:18.900914 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df3b052-e0d8-4f30-9c58-2fb9473403fc service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:48:18.901319 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df3b052-e0d8-4f30-9c58-2fb9473403fc service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:48:18.901593 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df3b052-e0d8-4f30-9c58-2fb9473403fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:48:18.903462 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-4df3b052-e0d8-4f30-9c58-2fb9473403fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:48:18.969848 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df3b052-e0d8-4f30-9c58-2fb9473403fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:48:18.971372 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df3b052-e0d8-4f30-9c58-2fb9473403fc service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:48:18.981508 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1219/2441] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:48:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:48:19.695440 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d219d64-7e0d-4cf9-80b2-755032c44095 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:48:19.696009 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d219d64-7e0d-4cf9-80b2-755032c44095 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:48:19.696009 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d219d64-7e0d-4cf9-80b2-755032c44095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:48:19.727744 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-4d219d64-7e0d-4cf9-80b2-755032c44095 None None] MFA Rules not processed for user `61388384a318473e987ab06426509b69`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:48:19.728984 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-4d219d64-7e0d-4cf9-80b2-755032c44095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:48:19.736637 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1223/2442] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:48:19 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:48:20.072136 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df3b052-e0d8-4f30-9c58-2fb9473403fc service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:48:20.081423 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df3b052-e0d8-4f30-9c58-2fb9473403fc service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:48:20.085245 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9211cae-d0a9-42f8-9bcb-bcebc88c1b64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:48:20.086676 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9211cae-d0a9-42f8-9bcb-bcebc88c1b64 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:48:20.086998 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9211cae-d0a9-42f8-9bcb-bcebc88c1b64 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:48:20.087271 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9211cae-d0a9-42f8-9bcb-bcebc88c1b64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:48:20.089074 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-f9211cae-d0a9-42f8-9bcb-bcebc88c1b64 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:48:20.099323 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9211cae-d0a9-42f8-9bcb-bcebc88c1b64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:48:20.100792 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9211cae-d0a9-42f8-9bcb-bcebc88c1b64 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:48:20.115861 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1220/2443] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:48:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:48:21.470518 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d219d64-7e0d-4cf9-80b2-755032c44095 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:48:21.472947 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-4d219d64-7e0d-4cf9-80b2-755032c44095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:48:21.489564 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d219d64-7e0d-4cf9-80b2-755032c44095 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:48:21.495269 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3da930-18aa-415a-a2d6-b49a91ba7b27 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eef0f73297e04d62bc22acb790fad6ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:48:21.496059 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3da930-18aa-415a-a2d6-b49a91ba7b27 service placement] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:48:21.496405 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3da930-18aa-415a-a2d6-b49a91ba7b27 service placement] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:48:21.496754 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3da930-18aa-415a-a2d6-b49a91ba7b27 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:48:21.498637 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-1d3da930-18aa-415a-a2d6-b49a91ba7b27 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:48:21.512708 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3da930-18aa-415a-a2d6-b49a91ba7b27 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:48:21.514344 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3da930-18aa-415a-a2d6-b49a91ba7b27 service placement] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:48:21.526789 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1224/2444] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:48:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:48:30.113558 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=76 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:48:31.521712 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=53 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:48:34.617084 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9211cae-d0a9-42f8-9bcb-bcebc88c1b64 service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:48:34.630745 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9211cae-d0a9-42f8-9bcb-bcebc88c1b64 service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:48:34.637743 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cfb77b-c2e3-465c-bbee-9c7cae52cdb7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f43e2756b746a6825ca1a06b7d7c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:48:34.638380 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cfb77b-c2e3-465c-bbee-9c7cae52cdb7 service nova] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:48:34.638657 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cfb77b-c2e3-465c-bbee-9c7cae52cdb7 service nova] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:48:34.638930 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cfb77b-c2e3-465c-bbee-9c7cae52cdb7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:48:34.640696 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-b0cfb77b-c2e3-465c-bbee-9c7cae52cdb7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:48:34.653920 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cfb77b-c2e3-465c-bbee-9c7cae52cdb7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:48:34.655507 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cfb77b-c2e3-465c-bbee-9c7cae52cdb7 service nova] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:48:34.666274 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1221/2445] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:48:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:48:44.661932 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=6 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:49:43.780159 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3da930-18aa-415a-a2d6-b49a91ba7b27 service placement] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:49:43.808535 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3da930-18aa-415a-a2d6-b49a91ba7b27 service placement] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:49:43.812149 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082a33ce-7ca8-49cf-9ec1-f782aace84ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e79498873d4ed29a0e19bbea9b9214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:49:43.812736 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082a33ce-7ca8-49cf-9ec1-f782aace84ab service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:49:43.813008 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082a33ce-7ca8-49cf-9ec1-f782aace84ab service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:49:43.813273 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082a33ce-7ca8-49cf-9ec1-f782aace84ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:49:43.822066 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082a33ce-7ca8-49cf-9ec1-f782aace84ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:49:43.823468 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082a33ce-7ca8-49cf-9ec1-f782aace84ab service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:49:43.832201 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1225/2446] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:49:43 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:49:44.044787 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cfb77b-c2e3-465c-bbee-9c7cae52cdb7 service nova] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:49:44.070642 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cfb77b-c2e3-465c-bbee-9c7cae52cdb7 service nova] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:49:44.074528 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b275b00a-e6f8-4842-89a2-fad0d8c869d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e79498873d4ed29a0e19bbea9b9214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:49:44.075149 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b275b00a-e6f8-4842-89a2-fad0d8c869d0 service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:49:44.075436 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b275b00a-e6f8-4842-89a2-fad0d8c869d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:49:44.075872 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b275b00a-e6f8-4842-89a2-fad0d8c869d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:49:44.085722 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b275b00a-e6f8-4842-89a2-fad0d8c869d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:49:44.087162 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b275b00a-e6f8-4842-89a2-fad0d8c869d0 service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:49:44.117197 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1222/2447] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:49:44 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:49:53.828211 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=15 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:49:54.114337 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=18 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:50:17.017358 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb017fa-5a8c-4491-a61b-1df03123832c None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:50:17.017942 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb017fa-5a8c-4491-a61b-1df03123832c None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:50:17.018070 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb017fa-5a8c-4491-a61b-1df03123832c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:50:17.070521 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-5eb017fa-5a8c-4491-a61b-1df03123832c None None] MFA Rules not processed for user `21d0ab6d1a724547b8c643287df897da`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:50:17.071695 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-5eb017fa-5a8c-4491-a61b-1df03123832c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:50:17.078632 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1226/2448] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:50:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:50:17.393270 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b275b00a-e6f8-4842-89a2-fad0d8c869d0 service cinder] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:50:17.414058 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b275b00a-e6f8-4842-89a2-fad0d8c869d0 service cinder] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:50:17.417739 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1b800e-8c55-42a3-97dc-db1d05e8b399 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:50:17.418345 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1b800e-8c55-42a3-97dc-db1d05e8b399 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:50:17.418627 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1b800e-8c55-42a3-97dc-db1d05e8b399 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:50:17.418900 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1b800e-8c55-42a3-97dc-db1d05e8b399 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:50:17.420667 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-7d1b800e-8c55-42a3-97dc-db1d05e8b399 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:50:17.430837 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1b800e-8c55-42a3-97dc-db1d05e8b399 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:50:17.432294 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1b800e-8c55-42a3-97dc-db1d05e8b399 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:50:17.440281 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1223/2449] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:50:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:50:19.602959 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb017fa-5a8c-4491-a61b-1df03123832c None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:50:19.613082 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb017fa-5a8c-4491-a61b-1df03123832c None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:50:19.616932 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f9fba1-51f6-4fdf-9f51-28159e14636f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:50:19.617547 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f9fba1-51f6-4fdf-9f51-28159e14636f service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:50:19.617823 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f9fba1-51f6-4fdf-9f51-28159e14636f service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:50:19.618093 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f9fba1-51f6-4fdf-9f51-28159e14636f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:50:19.626110 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f9fba1-51f6-4fdf-9f51-28159e14636f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:50:19.627559 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f9fba1-51f6-4fdf-9f51-28159e14636f service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:50:19.636386 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1227/2450] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:50:19 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:50:20.536588 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1b800e-8c55-42a3-97dc-db1d05e8b399 service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:50:20.550646 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1b800e-8c55-42a3-97dc-db1d05e8b399 service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:50:20.557909 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd555ad0-2e20-4dd1-94f6-898f60a4273b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f43e2756b746a6825ca1a06b7d7c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:50:20.558501 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd555ad0-2e20-4dd1-94f6-898f60a4273b service nova] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:50:20.558701 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd555ad0-2e20-4dd1-94f6-898f60a4273b service nova] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:50:20.559160 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd555ad0-2e20-4dd1-94f6-898f60a4273b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:50:20.571671 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd555ad0-2e20-4dd1-94f6-898f60a4273b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:50:20.573259 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd555ad0-2e20-4dd1-94f6-898f60a4273b service nova] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:50:20.585563 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1224/2451] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:50:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:50:29.633172 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=21 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:50:30.580104 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=16 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:50:30.727965 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f9fba1-51f6-4fdf-9f51-28159e14636f service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:50:30.742346 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f9fba1-51f6-4fdf-9f51-28159e14636f service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:50:30.747973 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e069041-4adc-42ab-ad15-fb5015fde913 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f43e2756b746a6825ca1a06b7d7c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:50:30.748647 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e069041-4adc-42ab-ad15-fb5015fde913 service nova] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:50:30.748992 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e069041-4adc-42ab-ad15-fb5015fde913 service nova] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:50:30.749331 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e069041-4adc-42ab-ad15-fb5015fde913 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:50:30.751136 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-8e069041-4adc-42ab-ad15-fb5015fde913 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:50:30.764445 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e069041-4adc-42ab-ad15-fb5015fde913 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:50:30.766092 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e069041-4adc-42ab-ad15-fb5015fde913 service nova] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:50:30.777245 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1228/2452] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:50:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:50:40.772347 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=6 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:50:42.918109 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd555ad0-2e20-4dd1-94f6-898f60a4273b service nova] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:50:42.928059 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd555ad0-2e20-4dd1-94f6-898f60a4273b service nova] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:50:42.931872 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6e8fba-ffeb-4f48-9695-aa64442f1535 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:50:42.932462 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6e8fba-ffeb-4f48-9695-aa64442f1535 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:50:42.932736 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6e8fba-ffeb-4f48-9695-aa64442f1535 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:50:42.933005 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6e8fba-ffeb-4f48-9695-aa64442f1535 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:50:42.941366 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6e8fba-ffeb-4f48-9695-aa64442f1535 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:50:42.942756 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6e8fba-ffeb-4f48-9695-aa64442f1535 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:50:42.970282 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1225/2453] 10.222.0.61 () {60 vars in 1396 bytes} [Mon Aug 22 19:50:42 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3270 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:50:52.967695 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=13 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:50:59.336414 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf0f06c-00ac-4b43-a174-06ab0983a956 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:50:59.336963 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf0f06c-00ac-4b43-a174-06ab0983a956 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:50:59.336963 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf0f06c-00ac-4b43-a174-06ab0983a956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:50:59.358075 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-abf0f06c-00ac-4b43-a174-06ab0983a956 None None] MFA Rules not processed for user `61388384a318473e987ab06426509b69`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:50:59.359465 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-abf0f06c-00ac-4b43-a174-06ab0983a956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:50:59.367712 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1229/2454] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:50:59 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:51:09.352895 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:51:44.405516 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6e8fba-ffeb-4f48-9695-aa64442f1535 service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:51:44.415636 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6e8fba-ffeb-4f48-9695-aa64442f1535 service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:51:44.419286 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd5ea61-9890-4e33-b64c-cb4fd4780820 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e79498873d4ed29a0e19bbea9b9214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:51:44.419913 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd5ea61-9890-4e33-b64c-cb4fd4780820 service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:51:44.420189 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd5ea61-9890-4e33-b64c-cb4fd4780820 service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:51:44.420456 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd5ea61-9890-4e33-b64c-cb4fd4780820 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:51:44.429594 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd5ea61-9890-4e33-b64c-cb4fd4780820 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:51:44.431276 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd5ea61-9890-4e33-b64c-cb4fd4780820 service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:51:44.442299 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1226/2455] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:51:44 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:51:54.436988 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=6 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:51:57.524769 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf0f06c-00ac-4b43-a174-06ab0983a956 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:51:57.554112 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf0f06c-00ac-4b43-a174-06ab0983a956 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:51:57.559503 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbe3841-5a38-41d7-8cce-37857195c09e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f43e2756b746a6825ca1a06b7d7c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:51:57.560200 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbe3841-5a38-41d7-8cce-37857195c09e service nova] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:51:57.560475 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbe3841-5a38-41d7-8cce-37857195c09e service nova] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:51:57.560740 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbe3841-5a38-41d7-8cce-37857195c09e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:51:57.568504 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbe3841-5a38-41d7-8cce-37857195c09e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:51:57.570801 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbe3841-5a38-41d7-8cce-37857195c09e service nova] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:51:57.597488 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1230/2456] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:51:57 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:52:07.594918 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=20 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:52:14.819994 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd5ea61-9890-4e33-b64c-cb4fd4780820 service cinder] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:52:14.830105 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd5ea61-9890-4e33-b64c-cb4fd4780820 service cinder] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:52:14.833756 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e000001f-e39a-42ad-9ee7-9dd2d59913a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:52:14.834352 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e000001f-e39a-42ad-9ee7-9dd2d59913a5 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:52:14.834626 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e000001f-e39a-42ad-9ee7-9dd2d59913a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:52:14.835163 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e000001f-e39a-42ad-9ee7-9dd2d59913a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:52:14.845278 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e000001f-e39a-42ad-9ee7-9dd2d59913a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:52:14.846686 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e000001f-e39a-42ad-9ee7-9dd2d59913a5 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:52:14.857617 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1227/2457] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:52:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:52:18.366276 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbe3841-5a38-41d7-8cce-37857195c09e service nova] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:52:18.376009 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbe3841-5a38-41d7-8cce-37857195c09e service nova] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:52:18.379568 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a926f2b-d785-4f6f-96e8-1fc69b513869 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:52:18.380195 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a926f2b-d785-4f6f-96e8-1fc69b513869 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:52:18.380472 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a926f2b-d785-4f6f-96e8-1fc69b513869 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:52:18.380741 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a926f2b-d785-4f6f-96e8-1fc69b513869 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:52:18.391403 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a926f2b-d785-4f6f-96e8-1fc69b513869 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:52:18.392853 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a926f2b-d785-4f6f-96e8-1fc69b513869 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:52:18.403842 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1231/2458] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:52:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:52:24.852750 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=6 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:52:28.398916 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=6 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:52:48.259526 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e000001f-e39a-42ad-9ee7-9dd2d59913a5 service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:52:48.326455 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e000001f-e39a-42ad-9ee7-9dd2d59913a5 service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:52:48.331473 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4605577f-2073-4b79-bc02-c9eb29ae2170 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eef0f73297e04d62bc22acb790fad6ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:52:48.332124 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4605577f-2073-4b79-bc02-c9eb29ae2170 service placement] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:52:48.332404 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4605577f-2073-4b79-bc02-c9eb29ae2170 service placement] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:52:48.332678 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4605577f-2073-4b79-bc02-c9eb29ae2170 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:52:48.334819 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-4605577f-2073-4b79-bc02-c9eb29ae2170 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:52:48.346723 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4605577f-2073-4b79-bc02-c9eb29ae2170 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:52:48.348332 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4605577f-2073-4b79-bc02-c9eb29ae2170 service placement] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:52:48.358072 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1228/2459] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:52:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:52:58.353909 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=32 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:55:42.505719 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a926f2b-d785-4f6f-96e8-1fc69b513869 service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:55:42.516711 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a926f2b-d785-4f6f-96e8-1fc69b513869 service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:55:42.520446 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879e9ec6-916e-4e1c-9982-e98464c7b893 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e79498873d4ed29a0e19bbea9b9214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:55:42.521094 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879e9ec6-916e-4e1c-9982-e98464c7b893 service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:55:42.521429 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879e9ec6-916e-4e1c-9982-e98464c7b893 service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:55:42.522303 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879e9ec6-916e-4e1c-9982-e98464c7b893 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:55:42.524359 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-879e9ec6-916e-4e1c-9982-e98464c7b893 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:55:42.612135 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879e9ec6-916e-4e1c-9982-e98464c7b893 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:55:42.613470 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879e9ec6-916e-4e1c-9982-e98464c7b893 service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:55:42.632492 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1232/2460] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:55:42 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:55:42.691074 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4605577f-2073-4b79-bc02-c9eb29ae2170 service placement] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:55:42.701448 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4605577f-2073-4b79-bc02-c9eb29ae2170 service placement] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:55:42.705009 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1f6286-6530-41f3-8adf-cb7920323561 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e79498873d4ed29a0e19bbea9b9214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:55:42.705631 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1f6286-6530-41f3-8adf-cb7920323561 service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:55:42.705899 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1f6286-6530-41f3-8adf-cb7920323561 service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:55:42.706163 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1f6286-6530-41f3-8adf-cb7920323561 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:55:42.708054 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-2c1f6286-6530-41f3-8adf-cb7920323561 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:55:42.779484 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1f6286-6530-41f3-8adf-cb7920323561 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:55:42.780936 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1f6286-6530-41f3-8adf-cb7920323561 service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:55:42.789854 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1229/2461] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:55:42 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:55:43.517004 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879e9ec6-916e-4e1c-9982-e98464c7b893 service cinder] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:55:43.518412 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-879e9ec6-916e-4e1c-9982-e98464c7b893 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:55:43.588626 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879e9ec6-916e-4e1c-9982-e98464c7b893 service cinder] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:55:43.592263 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94aaa08-7e7b-44d4-8e53-67c2d558644d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e79498873d4ed29a0e19bbea9b9214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:55:43.592872 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94aaa08-7e7b-44d4-8e53-67c2d558644d service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:55:43.593150 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94aaa08-7e7b-44d4-8e53-67c2d558644d service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:55:43.593424 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94aaa08-7e7b-44d4-8e53-67c2d558644d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:55:43.595300 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-a94aaa08-7e7b-44d4-8e53-67c2d558644d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:55:43.670055 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94aaa08-7e7b-44d4-8e53-67c2d558644d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:55:43.671511 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94aaa08-7e7b-44d4-8e53-67c2d558644d service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:55:43.681998 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1233/2462] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:55:43 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:55:44.931024 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1f6286-6530-41f3-8adf-cb7920323561 service cinder] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:55:44.932774 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-2c1f6286-6530-41f3-8adf-cb7920323561 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:55:45.022077 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1f6286-6530-41f3-8adf-cb7920323561 service cinder] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:55:45.025812 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84290ae-fefc-4725-9c1d-6805e39c9d1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:55:45.026467 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84290ae-fefc-4725-9c1d-6805e39c9d1c service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:55:45.026750 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84290ae-fefc-4725-9c1d-6805e39c9d1c service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:55:45.027179 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84290ae-fefc-4725-9c1d-6805e39c9d1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:55:45.028992 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-b84290ae-fefc-4725-9c1d-6805e39c9d1c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:55:45.077129 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84290ae-fefc-4725-9c1d-6805e39c9d1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:55:45.078695 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84290ae-fefc-4725-9c1d-6805e39c9d1c service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:55:45.100630 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1230/2463] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:55:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:55:46.234355 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94aaa08-7e7b-44d4-8e53-67c2d558644d service cinder] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:55:46.238797 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:55:46.246099 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94aaa08-7e7b-44d4-8e53-67c2d558644d service cinder] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:55:46.249794 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96507ec2-011a-4039-bb49-6793c4f3ff04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:55:46.250411 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96507ec2-011a-4039-bb49-6793c4f3ff04 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:55:46.250687 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96507ec2-011a-4039-bb49-6793c4f3ff04 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:55:46.250964 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96507ec2-011a-4039-bb49-6793c4f3ff04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:55:46.252853 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-96507ec2-011a-4039-bb49-6793c4f3ff04 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:55:46.265670 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96507ec2-011a-4039-bb49-6793c4f3ff04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:55:46.267038 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96507ec2-011a-4039-bb49-6793c4f3ff04 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:55:46.277578 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1234/2464] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:55:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:55:47.127577 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84290ae-fefc-4725-9c1d-6805e39c9d1c service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:55:47.129225 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-b84290ae-fefc-4725-9c1d-6805e39c9d1c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:55:47.195314 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:55:47.232830 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84290ae-fefc-4725-9c1d-6805e39c9d1c service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:55:47.239689 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d73a48b-170a-4308-89c3-74e5da219d35 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f43e2756b746a6825ca1a06b7d7c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:55:47.241227 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d73a48b-170a-4308-89c3-74e5da219d35 service nova] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:55:47.241711 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d73a48b-170a-4308-89c3-74e5da219d35 service nova] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:55:47.242037 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d73a48b-170a-4308-89c3-74e5da219d35 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:55:47.244470 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-7d73a48b-170a-4308-89c3-74e5da219d35 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:55:47.258645 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d73a48b-170a-4308-89c3-74e5da219d35 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:55:47.260461 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d73a48b-170a-4308-89c3-74e5da219d35 service nova] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:55:47.276260 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1231/2465] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:55:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:55:49.119331 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96507ec2-011a-4039-bb49-6793c4f3ff04 service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:55:49.145142 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96507ec2-011a-4039-bb49-6793c4f3ff04 service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:55:49.155267 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e00cf7b-87e0-4b0e-817f-4e73b1eb8252 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eef0f73297e04d62bc22acb790fad6ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:55:49.156175 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e00cf7b-87e0-4b0e-817f-4e73b1eb8252 service placement] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:55:49.156596 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e00cf7b-87e0-4b0e-817f-4e73b1eb8252 service placement] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:55:49.157037 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e00cf7b-87e0-4b0e-817f-4e73b1eb8252 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:55:49.160992 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-5e00cf7b-87e0-4b0e-817f-4e73b1eb8252 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:55:49.232470 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e00cf7b-87e0-4b0e-817f-4e73b1eb8252 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:55:49.234668 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d73a48b-170a-4308-89c3-74e5da219d35 service nova] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:55:49.236482 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e00cf7b-87e0-4b0e-817f-4e73b1eb8252 service placement] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:55:49.274642 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d73a48b-170a-4308-89c3-74e5da219d35 service nova] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:55:49.287521 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f06ffb-b906-47e3-bfd4-887d8a869fea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:55:49.288235 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f06ffb-b906-47e3-bfd4-887d8a869fea service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:55:49.288575 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f06ffb-b906-47e3-bfd4-887d8a869fea service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:55:49.289504 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f06ffb-b906-47e3-bfd4-887d8a869fea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:55:49.294233 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1235/2466] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:55:49 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:55:49.295046 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-86f06ffb-b906-47e3-bfd4-887d8a869fea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:55:49.295439 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e00cf7b-87e0-4b0e-817f-4e73b1eb8252 service placement] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:55:49.299491 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-5e00cf7b-87e0-4b0e-817f-4e73b1eb8252 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:55:49.319786 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f06ffb-b906-47e3-bfd4-887d8a869fea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:55:49.321793 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f06ffb-b906-47e3-bfd4-887d8a869fea service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:55:49.343046 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e00cf7b-87e0-4b0e-817f-4e73b1eb8252 service placement] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:55:49.348121 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1232/2467] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:55:49 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:55:49.356005 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adb13b5-d9c2-405e-bbf8-c27bbc1570c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:55:49.356847 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adb13b5-d9c2-405e-bbf8-c27bbc1570c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:55:49.357609 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adb13b5-d9c2-405e-bbf8-c27bbc1570c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:55:49.358042 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adb13b5-d9c2-405e-bbf8-c27bbc1570c9 admin admin] PATH_INFO: `/v3/users/21d0ab6d1a724547b8c643287df897da` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:55:49.360237 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adb13b5-d9c2-405e-bbf8-c27bbc1570c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=21d0ab6d1a724547b8c643287df897da)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:55:49.362833 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adb13b5-d9c2-405e-bbf8-c27bbc1570c9 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:55:49.483065 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-4adb13b5-d9c2-405e-bbf8-c27bbc1570c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21d0ab6d1a724547b8c643287df897da'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:55:49.490552 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-4adb13b5-d9c2-405e-bbf8-c27bbc1570c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21d0ab6d1a724547b8c643287df897da'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:55:49.499559 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-4adb13b5-d9c2-405e-bbf8-c27bbc1570c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21d0ab6d1a724547b8c643287df897da'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:55:49.509055 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-4adb13b5-d9c2-405e-bbf8-c27bbc1570c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21d0ab6d1a724547b8c643287df897da'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:55:49.533435 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-4adb13b5-d9c2-405e-bbf8-c27bbc1570c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21d0ab6d1a724547b8c643287df897da'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:55:49.539910 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1236/2468] 10.222.0.61 () {58 vars in 1224 bytes} [Mon Aug 22 19:55:49 2022] DELETE /identity/v3/users/21d0ab6d1a724547b8c643287df897da => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:55:49.724230 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f06ffb-b906-47e3-bfd4-887d8a869fea service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:55:49.736665 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f06ffb-b906-47e3-bfd4-887d8a869fea service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:55:49.817253 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ec6625-8eb5-449b-8c69-a0b780d7a38a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:55:49.817902 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ec6625-8eb5-449b-8c69-a0b780d7a38a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:55:49.818226 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ec6625-8eb5-449b-8c69-a0b780d7a38a admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:55:49.818802 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ec6625-8eb5-449b-8c69-a0b780d7a38a admin admin] PATH_INFO: `/v3/projects/ca75caa807b445f486899173277b5051` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:55:49.820350 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ec6625-8eb5-449b-8c69-a0b780d7a38a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca75caa807b445f486899173277b5051)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:55:49.822624 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ec6625-8eb5-449b-8c69-a0b780d7a38a admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:55:49.877726 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-41ec6625-8eb5-449b-8c69-a0b780d7a38a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca75caa807b445f486899173277b5051', 'request_id': 'req-41ec6625-8eb5-449b-8c69-a0b780d7a38a'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:55:49.887343 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-41ec6625-8eb5-449b-8c69-a0b780d7a38a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca75caa807b445f486899173277b5051', 'request_id': 'req-41ec6625-8eb5-449b-8c69-a0b780d7a38a'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:55:49.895476 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-41ec6625-8eb5-449b-8c69-a0b780d7a38a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca75caa807b445f486899173277b5051', 'request_id': 'req-41ec6625-8eb5-449b-8c69-a0b780d7a38a'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:55:49.906539 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-41ec6625-8eb5-449b-8c69-a0b780d7a38a admin admin] The token cache is being invalidate because project ca75caa807b445f486899173277b5051 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 19:55:49.906829 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-41ec6625-8eb5-449b-8c69-a0b780d7a38a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:55:49.907565 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-41ec6625-8eb5-449b-8c69-a0b780d7a38a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:55:49.909668 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1233/2469] 10.222.0.61 () {58 vars in 1233 bytes} [Mon Aug 22 19:55:49 2022] DELETE /identity/v3/projects/ca75caa807b445f486899173277b5051 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:55:49.916193 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9690086b-2521-42ea-b15f-47d968862194 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:55:49.916561 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9690086b-2521-42ea-b15f-47d968862194 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:55:49.916910 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9690086b-2521-42ea-b15f-47d968862194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:55:49.976390 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-9690086b-2521-42ea-b15f-47d968862194 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:55:49.977630 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-9690086b-2521-42ea-b15f-47d968862194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:55:50.041526 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1237/2470] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:55:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 19:55:50.050472 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1402efcc-b7c6-48dd-b0b5-7f2daf838bef None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:55:50.050755 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1402efcc-b7c6-48dd-b0b5-7f2daf838bef None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:55:50.051038 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1402efcc-b7c6-48dd-b0b5-7f2daf838bef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:55:50.071961 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-1402efcc-b7c6-48dd-b0b5-7f2daf838bef None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:55:50.073196 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-1402efcc-b7c6-48dd-b0b5-7f2daf838bef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:55:50.081617 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1234/2471] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:55:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:55:50.085973 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9690086b-2521-42ea-b15f-47d968862194 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:55:50.096626 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9690086b-2521-42ea-b15f-47d968862194 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:55:50.104345 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eef17c-897c-4d20-956f-a9c1806bf69b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:55:50.105084 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eef17c-897c-4d20-956f-a9c1806bf69b admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:55:50.105360 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eef17c-897c-4d20-956f-a9c1806bf69b admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:55:50.105833 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eef17c-897c-4d20-956f-a9c1806bf69b admin admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:55:50.106895 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20eef17c-897c-4d20-956f-a9c1806bf69b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:55:50.107666 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20eef17c-897c-4d20-956f-a9c1806bf69b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:55:50.109202 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20eef17c-897c-4d20-956f-a9c1806bf69b admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:55:50.123211 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1238/2472] 10.222.0.61 () {58 vars in 1153 bytes} [Mon Aug 22 19:55:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:55:50.127443 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1402efcc-b7c6-48dd-b0b5-7f2daf838bef None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:55:50.137533 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1402efcc-b7c6-48dd-b0b5-7f2daf838bef None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:55:50.147420 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9669d53-d36e-4d5e-a7d5-b1de092a95ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:55:50.148076 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9669d53-d36e-4d5e-a7d5-b1de092a95ec admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:55:50.148351 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9669d53-d36e-4d5e-a7d5-b1de092a95ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:55:50.148619 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9669d53-d36e-4d5e-a7d5-b1de092a95ec admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:55:50.149287 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9669d53-d36e-4d5e-a7d5-b1de092a95ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:55:50.151547 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9669d53-d36e-4d5e-a7d5-b1de092a95ec admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:55:50.202992 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1235/2473] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 19:55:50 2022] POST /identity/v3/projects => generated 373 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 19:55:50.204558 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eef17c-897c-4d20-956f-a9c1806bf69b admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:55:50.212109 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:55:50.216997 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eef17c-897c-4d20-956f-a9c1806bf69b admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:55:50.284036 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60209df-0545-4dfa-9d4b-b06d645614f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:55:50.284729 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60209df-0545-4dfa-9d4b-b06d645614f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:55:50.285086 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60209df-0545-4dfa-9d4b-b06d645614f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:55:50.285433 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60209df-0545-4dfa-9d4b-b06d645614f4 admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:55:50.286131 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60209df-0545-4dfa-9d4b-b06d645614f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:55:50.288321 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60209df-0545-4dfa-9d4b-b06d645614f4 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:55:50.314324 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1239/2474] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 19:55:50 2022] POST /identity/v3/users => generated 287 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:55:50.319523 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9669d53-d36e-4d5e-a7d5-b1de092a95ec admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:55:50.326744 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:55:50.334802 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9669d53-d36e-4d5e-a7d5-b1de092a95ec admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:55:50.343298 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dfdc81-50e7-42b0-b08d-e6c04f41fd77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:55:50.343944 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dfdc81-50e7-42b0-b08d-e6c04f41fd77 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:55:50.344222 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dfdc81-50e7-42b0-b08d-e6c04f41fd77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:55:50.344494 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dfdc81-50e7-42b0-b08d-e6c04f41fd77 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:55:50.345142 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dfdc81-50e7-42b0-b08d-e6c04f41fd77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:55:50.346731 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dfdc81-50e7-42b0-b08d-e6c04f41fd77 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:55:50.356940 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1236/2475] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:55:50 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:55:50.362133 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60209df-0545-4dfa-9d4b-b06d645614f4 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:55:50.372810 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60209df-0545-4dfa-9d4b-b06d645614f4 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:55:50.377495 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e9be52-d185-4e33-a738-7a9ff723ad99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:55:50.378097 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e9be52-d185-4e33-a738-7a9ff723ad99 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:55:50.378377 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e9be52-d185-4e33-a738-7a9ff723ad99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:55:50.378660 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e9be52-d185-4e33-a738-7a9ff723ad99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:55:50.379285 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e9be52-d185-4e33-a738-7a9ff723ad99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:55:50.381497 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e9be52-d185-4e33-a738-7a9ff723ad99 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:55:50.392011 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1240/2476] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:55:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:55:50.396000 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dfdc81-50e7-42b0-b08d-e6c04f41fd77 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:55:50.405927 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dfdc81-50e7-42b0-b08d-e6c04f41fd77 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:55:50.410576 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5435b2a2-c46d-4fde-a046-df05d47227f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:55:50.411219 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5435b2a2-c46d-4fde-a046-df05d47227f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:55:50.411503 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5435b2a2-c46d-4fde-a046-df05d47227f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:55:50.411859 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5435b2a2-c46d-4fde-a046-df05d47227f5 admin admin] PATH_INFO: `/v3/projects/ed77f65b9c474e3288f3e83cf54d5e5f/users/1c37c248654a417191ced0acb2d767ad/roles/74a40bc819cb42eea7e966a4b2a016f1` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:55:50.427089 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5435b2a2-c46d-4fde-a046-df05d47227f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed77f65b9c474e3288f3e83cf54d5e5f, user_id=1c37c248654a417191ced0acb2d767ad, role_id=74a40bc819cb42eea7e966a4b2a016f1)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:55:50.430306 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5435b2a2-c46d-4fde-a046-df05d47227f5 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:55:50.446082 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1237/2477] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:55:50 2022] PUT /identity/v3/projects/ed77f65b9c474e3288f3e83cf54d5e5f/users/1c37c248654a417191ced0acb2d767ad/roles/74a40bc819cb42eea7e966a4b2a016f1 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:55:50.456833 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a86cd10-1f50-4093-9259-85b391088cf7 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:55:50.456833 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a86cd10-1f50-4093-9259-85b391088cf7 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:55:50.457069 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a86cd10-1f50-4093-9259-85b391088cf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:55:50.484650 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-5a86cd10-1f50-4093-9259-85b391088cf7 None None] MFA Rules not processed for user `1c37c248654a417191ced0acb2d767ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:55:50.485933 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-5a86cd10-1f50-4093-9259-85b391088cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:55:50.551119 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1241/2478] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:55:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:55:50.563406 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5435b2a2-c46d-4fde-a046-df05d47227f5 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:55:50.565256 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-5435b2a2-c46d-4fde-a046-df05d47227f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:55:50.639747 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5435b2a2-c46d-4fde-a046-df05d47227f5 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:55:50.645529 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05985b94-079d-407d-8695-0978276fc0f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f43e2756b746a6825ca1a06b7d7c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:55:50.646091 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05985b94-079d-407d-8695-0978276fc0f5 service nova] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:55:50.646452 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05985b94-079d-407d-8695-0978276fc0f5 service nova] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:55:50.646730 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05985b94-079d-407d-8695-0978276fc0f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:55:50.654682 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05985b94-079d-407d-8695-0978276fc0f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:55:50.656288 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05985b94-079d-407d-8695-0978276fc0f5 service nova] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:55:50.665693 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1238/2479] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:55:50 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:55:53.349149 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a86cd10-1f50-4093-9259-85b391088cf7 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:55:53.412064 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a86cd10-1f50-4093-9259-85b391088cf7 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:55:53.417698 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e082eb1-2204-468e-bd10-8cfb43fd8536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:55:53.418480 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e082eb1-2204-468e-bd10-8cfb43fd8536 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:55:53.418717 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e082eb1-2204-468e-bd10-8cfb43fd8536 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:55:53.418885 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e082eb1-2204-468e-bd10-8cfb43fd8536 admin admin] PATH_INFO: `/v3/users/1c37c248654a417191ced0acb2d767ad` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:55:53.420073 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e082eb1-2204-468e-bd10-8cfb43fd8536 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c37c248654a417191ced0acb2d767ad)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:55:53.422093 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e082eb1-2204-468e-bd10-8cfb43fd8536 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:55:53.440925 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:55:53.480216 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-7e082eb1-2204-468e-bd10-8cfb43fd8536 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c37c248654a417191ced0acb2d767ad'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:55:53.480864 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-7e082eb1-2204-468e-bd10-8cfb43fd8536 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c37c248654a417191ced0acb2d767ad'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:55:53.489387 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-7e082eb1-2204-468e-bd10-8cfb43fd8536 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c37c248654a417191ced0acb2d767ad'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:55:53.498048 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-7e082eb1-2204-468e-bd10-8cfb43fd8536 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c37c248654a417191ced0acb2d767ad'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:55:53.510894 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-7e082eb1-2204-468e-bd10-8cfb43fd8536 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c37c248654a417191ced0acb2d767ad'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:55:53.517129 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1242/2480] 10.222.0.61 () {58 vars in 1224 bytes} [Mon Aug 22 19:55:53 2022] DELETE /identity/v3/users/1c37c248654a417191ced0acb2d767ad => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:55:53.521480 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d97a02-515f-405c-a0c6-318bbeb1150a None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:55:53.522013 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d97a02-515f-405c-a0c6-318bbeb1150a None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:55:53.522188 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d97a02-515f-405c-a0c6-318bbeb1150a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:55:53.542663 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-72d97a02-515f-405c-a0c6-318bbeb1150a None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:55:53.543917 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-72d97a02-515f-405c-a0c6-318bbeb1150a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:55:53.607504 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1239/2481] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:55:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:55:53.618037 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e082eb1-2204-468e-bd10-8cfb43fd8536 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:55:53.619601 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-7e082eb1-2204-468e-bd10-8cfb43fd8536 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:55:53.676786 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e082eb1-2204-468e-bd10-8cfb43fd8536 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:55:53.680923 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3cbd5d-b5c6-4290-8e59-0e35bf825bf5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:55:53.681524 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3cbd5d-b5c6-4290-8e59-0e35bf825bf5 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:55:53.681809 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3cbd5d-b5c6-4290-8e59-0e35bf825bf5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:55:53.682082 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3cbd5d-b5c6-4290-8e59-0e35bf825bf5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:55:53.691004 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3cbd5d-b5c6-4290-8e59-0e35bf825bf5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:55:53.692457 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3cbd5d-b5c6-4290-8e59-0e35bf825bf5 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:55:53.701746 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1243/2482] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:55:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:55:53.983102 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d97a02-515f-405c-a0c6-318bbeb1150a None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:55:53.984685 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-72d97a02-515f-405c-a0c6-318bbeb1150a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:55:53.998896 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d97a02-515f-405c-a0c6-318bbeb1150a None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:55:54.004011 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373b0055-c233-46b4-a7d5-7dfe68d70bce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:55:54.004648 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373b0055-c233-46b4-a7d5-7dfe68d70bce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:55:54.004926 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373b0055-c233-46b4-a7d5-7dfe68d70bce admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:55:54.005193 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373b0055-c233-46b4-a7d5-7dfe68d70bce admin admin] PATH_INFO: `/v3/projects/ed77f65b9c474e3288f3e83cf54d5e5f` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:55:54.007040 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373b0055-c233-46b4-a7d5-7dfe68d70bce admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed77f65b9c474e3288f3e83cf54d5e5f)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:55:54.009034 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373b0055-c233-46b4-a7d5-7dfe68d70bce admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:55:54.056902 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-373b0055-c233-46b4-a7d5-7dfe68d70bce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed77f65b9c474e3288f3e83cf54d5e5f', 'request_id': 'req-373b0055-c233-46b4-a7d5-7dfe68d70bce'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:55:54.058588 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:55:54.065483 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-373b0055-c233-46b4-a7d5-7dfe68d70bce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed77f65b9c474e3288f3e83cf54d5e5f', 'request_id': 'req-373b0055-c233-46b4-a7d5-7dfe68d70bce'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:55:54.072985 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-373b0055-c233-46b4-a7d5-7dfe68d70bce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed77f65b9c474e3288f3e83cf54d5e5f', 'request_id': 'req-373b0055-c233-46b4-a7d5-7dfe68d70bce'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:55:54.083774 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-373b0055-c233-46b4-a7d5-7dfe68d70bce admin admin] The token cache is being invalidate because project ed77f65b9c474e3288f3e83cf54d5e5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 19:55:54.084062 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-373b0055-c233-46b4-a7d5-7dfe68d70bce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:55:54.085235 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-373b0055-c233-46b4-a7d5-7dfe68d70bce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:55:54.086745 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1240/2483] 10.222.0.61 () {58 vars in 1233 bytes} [Mon Aug 22 19:55:53 2022] DELETE /identity/v3/projects/ed77f65b9c474e3288f3e83cf54d5e5f => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:55:54.091767 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875690d1-5b0d-4bea-813c-661bfb01a268 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:55:54.092057 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875690d1-5b0d-4bea-813c-661bfb01a268 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:55:54.092330 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875690d1-5b0d-4bea-813c-661bfb01a268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:55:54.112243 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-875690d1-5b0d-4bea-813c-661bfb01a268 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:55:54.113370 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-875690d1-5b0d-4bea-813c-661bfb01a268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:55:54.174384 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1244/2484] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:55:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:55:54.183937 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9250dbdb-6afd-4eb4-91d8-ddf60ef8897d None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:55:54.184129 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9250dbdb-6afd-4eb4-91d8-ddf60ef8897d None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:55:54.184402 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9250dbdb-6afd-4eb4-91d8-ddf60ef8897d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:55:54.203625 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-9250dbdb-6afd-4eb4-91d8-ddf60ef8897d None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:55:54.204852 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-9250dbdb-6afd-4eb4-91d8-ddf60ef8897d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:55:54.213440 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1241/2485] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:55:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:55:54.217551 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875690d1-5b0d-4bea-813c-661bfb01a268 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:55:54.228357 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875690d1-5b0d-4bea-813c-661bfb01a268 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:55:54.233928 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2704a4-d5e6-40ca-a6f4-6e10a0a88fd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:55:54.234526 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2704a4-d5e6-40ca-a6f4-6e10a0a88fd1 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:55:54.234832 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2704a4-d5e6-40ca-a6f4-6e10a0a88fd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:55:54.235167 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2704a4-d5e6-40ca-a6f4-6e10a0a88fd1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:55:54.235928 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2704a4-d5e6-40ca-a6f4-6e10a0a88fd1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:55:54.236237 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2704a4-d5e6-40ca-a6f4-6e10a0a88fd1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:55:54.237851 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2704a4-d5e6-40ca-a6f4-6e10a0a88fd1 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:55:54.249368 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1245/2486] 10.222.0.61 () {58 vars in 1153 bytes} [Mon Aug 22 19:55:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:55:54.253724 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9250dbdb-6afd-4eb4-91d8-ddf60ef8897d None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:55:54.263269 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9250dbdb-6afd-4eb4-91d8-ddf60ef8897d None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:55:54.267963 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f6cfbe-c0f0-4deb-b2dc-ec2e6cc08c73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:55:54.268554 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f6cfbe-c0f0-4deb-b2dc-ec2e6cc08c73 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:55:54.268832 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f6cfbe-c0f0-4deb-b2dc-ec2e6cc08c73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:55:54.269101 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f6cfbe-c0f0-4deb-b2dc-ec2e6cc08c73 admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:55:54.269740 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f6cfbe-c0f0-4deb-b2dc-ec2e6cc08c73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:55:54.272716 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f6cfbe-c0f0-4deb-b2dc-ec2e6cc08c73 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:55:54.301425 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1242/2487] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 19:55:54 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:55:54.305597 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2704a4-d5e6-40ca-a6f4-6e10a0a88fd1 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:55:54.315953 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2704a4-d5e6-40ca-a6f4-6e10a0a88fd1 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:55:54.339487 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:55:54.379678 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6ada83-67f4-489b-890d-0b815af8b379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:55:54.380299 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6ada83-67f4-489b-890d-0b815af8b379 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:55:54.380542 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6ada83-67f4-489b-890d-0b815af8b379 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:55:54.380807 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6ada83-67f4-489b-890d-0b815af8b379 admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:55:54.381477 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6ada83-67f4-489b-890d-0b815af8b379 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:55:54.383373 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6ada83-67f4-489b-890d-0b815af8b379 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:55:54.404184 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1246/2488] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 19:55:54 2022] POST /identity/v3/users => generated 287 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 19:55:54.408447 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f6cfbe-c0f0-4deb-b2dc-ec2e6cc08c73 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:55:54.419304 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f6cfbe-c0f0-4deb-b2dc-ec2e6cc08c73 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:55:54.424515 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2655b9c9-66fc-4700-8024-76295b83e4af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:55:54.425119 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2655b9c9-66fc-4700-8024-76295b83e4af admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:55:54.425400 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2655b9c9-66fc-4700-8024-76295b83e4af admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:55:54.425675 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2655b9c9-66fc-4700-8024-76295b83e4af admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:55:54.426324 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2655b9c9-66fc-4700-8024-76295b83e4af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:55:54.428838 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2655b9c9-66fc-4700-8024-76295b83e4af admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:55:54.438898 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1243/2489] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:55:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:55:54.442797 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6ada83-67f4-489b-890d-0b815af8b379 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:55:54.453965 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6ada83-67f4-489b-890d-0b815af8b379 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:55:54.458863 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c730501-3e7b-4992-ba93-ee1f19b54b23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:55:54.460107 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c730501-3e7b-4992-ba93-ee1f19b54b23 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:55:54.460389 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c730501-3e7b-4992-ba93-ee1f19b54b23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:55:54.460657 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c730501-3e7b-4992-ba93-ee1f19b54b23 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:55:54.461300 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c730501-3e7b-4992-ba93-ee1f19b54b23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:55:54.462894 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c730501-3e7b-4992-ba93-ee1f19b54b23 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:55:54.473526 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1247/2490] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:55:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:55:54.477855 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2655b9c9-66fc-4700-8024-76295b83e4af admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:55:54.487968 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2655b9c9-66fc-4700-8024-76295b83e4af admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:55:54.492715 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3d0884-2fc1-4508-a6e6-3ee4241e89c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:55:54.493366 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3d0884-2fc1-4508-a6e6-3ee4241e89c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:55:54.493646 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3d0884-2fc1-4508-a6e6-3ee4241e89c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:55:54.493920 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3d0884-2fc1-4508-a6e6-3ee4241e89c2 admin admin] PATH_INFO: `/v3/projects/2346c9f539ad4a9882738574eeda51bf/users/0e24963c0982447babdc41a6bdc4cb0b/roles/74a40bc819cb42eea7e966a4b2a016f1` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:55:54.510878 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3d0884-2fc1-4508-a6e6-3ee4241e89c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2346c9f539ad4a9882738574eeda51bf, user_id=0e24963c0982447babdc41a6bdc4cb0b, role_id=74a40bc819cb42eea7e966a4b2a016f1)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:55:54.513552 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3d0884-2fc1-4508-a6e6-3ee4241e89c2 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:55:54.529911 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1244/2491] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:55:54 2022] PUT /identity/v3/projects/2346c9f539ad4a9882738574eeda51bf/users/0e24963c0982447babdc41a6bdc4cb0b/roles/74a40bc819cb42eea7e966a4b2a016f1 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 19:55:54.598171 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18e7f39-4fe9-4f14-8aa3-602b104b8b24 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:55:54.598480 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18e7f39-4fe9-4f14-8aa3-602b104b8b24 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:55:54.598756 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18e7f39-4fe9-4f14-8aa3-602b104b8b24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:55:54.626311 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-c18e7f39-4fe9-4f14-8aa3-602b104b8b24 None None] MFA Rules not processed for user `0e24963c0982447babdc41a6bdc4cb0b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:55:54.627541 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-c18e7f39-4fe9-4f14-8aa3-602b104b8b24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:55:54.691561 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1248/2492] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:55:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:55:54.703336 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3d0884-2fc1-4508-a6e6-3ee4241e89c2 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:55:54.705063 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-dc3d0884-2fc1-4508-a6e6-3ee4241e89c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:55:54.779067 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3d0884-2fc1-4508-a6e6-3ee4241e89c2 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:55:54.784574 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f2971c-b7b7-49ac-a215-b66dad951f75 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f43e2756b746a6825ca1a06b7d7c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:55:54.785167 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f2971c-b7b7-49ac-a215-b66dad951f75 service nova] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:55:54.785441 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f2971c-b7b7-49ac-a215-b66dad951f75 service nova] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:55:54.785704 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f2971c-b7b7-49ac-a215-b66dad951f75 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:55:54.793257 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f2971c-b7b7-49ac-a215-b66dad951f75 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:55:54.795797 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f2971c-b7b7-49ac-a215-b66dad951f75 service nova] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:55:54.805272 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1245/2493] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:55:54 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:55:55.007150 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18e7f39-4fe9-4f14-8aa3-602b104b8b24 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:55:55.069833 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18e7f39-4fe9-4f14-8aa3-602b104b8b24 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:55:55.074369 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b506da-29e7-4e2d-8f03-63566e0b1319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:55:55.074991 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b506da-29e7-4e2d-8f03-63566e0b1319 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:55:55.075267 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b506da-29e7-4e2d-8f03-63566e0b1319 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:55:55.075537 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b506da-29e7-4e2d-8f03-63566e0b1319 admin admin] PATH_INFO: `/v3/users/0e24963c0982447babdc41a6bdc4cb0b` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:55:55.076686 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b506da-29e7-4e2d-8f03-63566e0b1319 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e24963c0982447babdc41a6bdc4cb0b)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:55:55.078581 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b506da-29e7-4e2d-8f03-63566e0b1319 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:55:55.112301 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:55:55.126995 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-96b506da-29e7-4e2d-8f03-63566e0b1319 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e24963c0982447babdc41a6bdc4cb0b'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:55:55.127675 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-96b506da-29e7-4e2d-8f03-63566e0b1319 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e24963c0982447babdc41a6bdc4cb0b'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:55:55.134844 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-96b506da-29e7-4e2d-8f03-63566e0b1319 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e24963c0982447babdc41a6bdc4cb0b'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:55:55.142268 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-96b506da-29e7-4e2d-8f03-63566e0b1319 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e24963c0982447babdc41a6bdc4cb0b'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:55:55.154648 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-96b506da-29e7-4e2d-8f03-63566e0b1319 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e24963c0982447babdc41a6bdc4cb0b'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:55:55.160587 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1249/2494] 10.222.0.61 () {58 vars in 1224 bytes} [Mon Aug 22 19:55:55 2022] DELETE /identity/v3/users/0e24963c0982447babdc41a6bdc4cb0b => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:55:55.165582 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee92bb8b-f8b4-46f5-af8f-ddcd2b693ab4 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:55:55.165860 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee92bb8b-f8b4-46f5-af8f-ddcd2b693ab4 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:55:55.166129 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee92bb8b-f8b4-46f5-af8f-ddcd2b693ab4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:55:55.185162 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-ee92bb8b-f8b4-46f5-af8f-ddcd2b693ab4 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:55:55.186397 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-ee92bb8b-f8b4-46f5-af8f-ddcd2b693ab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:55:55.237525 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:INSERT=3,keystone:DELETE=3,keystone:SELECT=94 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:55:55.249796 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1246/2495] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:55:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:55:55.259233 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b506da-29e7-4e2d-8f03-63566e0b1319 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:55:55.260689 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-96b506da-29e7-4e2d-8f03-63566e0b1319 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:55:55.317876 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b506da-29e7-4e2d-8f03-63566e0b1319 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:55:55.321742 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02464d41-fb0d-4fe0-a9aa-adc4f9da507e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:55:55.322358 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02464d41-fb0d-4fe0-a9aa-adc4f9da507e service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:55:55.322617 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02464d41-fb0d-4fe0-a9aa-adc4f9da507e service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:55:55.322890 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02464d41-fb0d-4fe0-a9aa-adc4f9da507e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:55:55.332707 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02464d41-fb0d-4fe0-a9aa-adc4f9da507e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:55:55.334699 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02464d41-fb0d-4fe0-a9aa-adc4f9da507e service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:55:55.345646 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1250/2496] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:55:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 19:55:55.638784 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee92bb8b-f8b4-46f5-af8f-ddcd2b693ab4 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:55:55.640567 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-ee92bb8b-f8b4-46f5-af8f-ddcd2b693ab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:55:55.654274 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee92bb8b-f8b4-46f5-af8f-ddcd2b693ab4 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:55:55.658938 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fcb39a-7ef3-41d4-a1db-bbde638b4d66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:55:55.659573 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fcb39a-7ef3-41d4-a1db-bbde638b4d66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:55:55.659908 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fcb39a-7ef3-41d4-a1db-bbde638b4d66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:55:55.660159 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fcb39a-7ef3-41d4-a1db-bbde638b4d66 admin admin] PATH_INFO: `/v3/projects/2346c9f539ad4a9882738574eeda51bf` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:55:55.661289 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fcb39a-7ef3-41d4-a1db-bbde638b4d66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2346c9f539ad4a9882738574eeda51bf)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:55:55.663878 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fcb39a-7ef3-41d4-a1db-bbde638b4d66 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:55:55.721532 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-94fcb39a-7ef3-41d4-a1db-bbde638b4d66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2346c9f539ad4a9882738574eeda51bf', 'request_id': 'req-94fcb39a-7ef3-41d4-a1db-bbde638b4d66'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:55:55.730079 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-94fcb39a-7ef3-41d4-a1db-bbde638b4d66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2346c9f539ad4a9882738574eeda51bf', 'request_id': 'req-94fcb39a-7ef3-41d4-a1db-bbde638b4d66'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:55:55.736450 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-94fcb39a-7ef3-41d4-a1db-bbde638b4d66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2346c9f539ad4a9882738574eeda51bf', 'request_id': 'req-94fcb39a-7ef3-41d4-a1db-bbde638b4d66'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:55:55.746624 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-94fcb39a-7ef3-41d4-a1db-bbde638b4d66 admin admin] The token cache is being invalidate because project 2346c9f539ad4a9882738574eeda51bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 19:55:55.746867 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-94fcb39a-7ef3-41d4-a1db-bbde638b4d66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:55:55.747471 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-94fcb39a-7ef3-41d4-a1db-bbde638b4d66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:55:55.748984 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1247/2497] 10.222.0.61 () {58 vars in 1233 bytes} [Mon Aug 22 19:55:55 2022] DELETE /identity/v3/projects/2346c9f539ad4a9882738574eeda51bf => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 19:55:55.754299 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51089933-bdd0-4d39-a85a-ea293a7008aa None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:55:55.754819 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51089933-bdd0-4d39-a85a-ea293a7008aa None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:55:55.754933 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51089933-bdd0-4d39-a85a-ea293a7008aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:55:55.777515 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-51089933-bdd0-4d39-a85a-ea293a7008aa None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:55:55.778743 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-51089933-bdd0-4d39-a85a-ea293a7008aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:55:55.841246 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1251/2498] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:55:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:55:55.848807 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c00230-7f79-4e3e-8b8f-476f7bf9ab46 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:55:55.849081 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c00230-7f79-4e3e-8b8f-476f7bf9ab46 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:55:55.849376 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c00230-7f79-4e3e-8b8f-476f7bf9ab46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:55:55.868474 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-08c00230-7f79-4e3e-8b8f-476f7bf9ab46 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:55:55.869681 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-08c00230-7f79-4e3e-8b8f-476f7bf9ab46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:55:55.877411 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1248/2499] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:55:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:55:55.882033 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51089933-bdd0-4d39-a85a-ea293a7008aa None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:55:55.891852 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51089933-bdd0-4d39-a85a-ea293a7008aa None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:55:55.896432 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb4e965-6e36-4699-b1ce-cba7a09ca5d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:55:55.897664 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb4e965-6e36-4699-b1ce-cba7a09ca5d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:55:55.897943 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb4e965-6e36-4699-b1ce-cba7a09ca5d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:55:55.898215 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb4e965-6e36-4699-b1ce-cba7a09ca5d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:55:55.898987 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb4e965-6e36-4699-b1ce-cba7a09ca5d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:55:55.899277 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb4e965-6e36-4699-b1ce-cba7a09ca5d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:55:55.900924 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb4e965-6e36-4699-b1ce-cba7a09ca5d6 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:55:55.912259 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1252/2500] 10.222.0.61 () {58 vars in 1153 bytes} [Mon Aug 22 19:55:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:55:55.916365 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c00230-7f79-4e3e-8b8f-476f7bf9ab46 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:55:55.925758 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c00230-7f79-4e3e-8b8f-476f7bf9ab46 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:55:55.930985 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f043b062-8d86-4cbe-8e09-cca4b957f281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:55:55.931576 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f043b062-8d86-4cbe-8e09-cca4b957f281 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:55:55.931886 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f043b062-8d86-4cbe-8e09-cca4b957f281 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:55:55.932157 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f043b062-8d86-4cbe-8e09-cca4b957f281 admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:55:55.932781 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f043b062-8d86-4cbe-8e09-cca4b957f281 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:55:55.934704 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f043b062-8d86-4cbe-8e09-cca4b957f281 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:55:55.962401 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1249/2501] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 19:55:55 2022] POST /identity/v3/projects => generated 363 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:55:55.966548 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb4e965-6e36-4699-b1ce-cba7a09ca5d6 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:55:55.976511 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb4e965-6e36-4699-b1ce-cba7a09ca5d6 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:55:56.013955 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:55:56.036070 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc465b9-c6f8-4c28-88d9-f9fe01ba7a97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:55:56.036680 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc465b9-c6f8-4c28-88d9-f9fe01ba7a97 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:55:56.036949 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc465b9-c6f8-4c28-88d9-f9fe01ba7a97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:55:56.037217 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc465b9-c6f8-4c28-88d9-f9fe01ba7a97 admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:55:56.037864 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc465b9-c6f8-4c28-88d9-f9fe01ba7a97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:55:56.039801 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc465b9-c6f8-4c28-88d9-f9fe01ba7a97 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:55:56.060135 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1253/2502] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 19:55:55 2022] POST /identity/v3/users => generated 282 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:55:56.064202 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f043b062-8d86-4cbe-8e09-cca4b957f281 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:55:56.074929 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f043b062-8d86-4cbe-8e09-cca4b957f281 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:55:56.079563 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e823322-5f1d-42ef-b1f3-9b317c5e6343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:55:56.080185 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e823322-5f1d-42ef-b1f3-9b317c5e6343 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:55:56.080462 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e823322-5f1d-42ef-b1f3-9b317c5e6343 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:55:56.080732 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e823322-5f1d-42ef-b1f3-9b317c5e6343 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:55:56.081374 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e823322-5f1d-42ef-b1f3-9b317c5e6343 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:55:56.083206 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e823322-5f1d-42ef-b1f3-9b317c5e6343 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:55:56.093599 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1250/2503] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:55:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:55:56.097432 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc465b9-c6f8-4c28-88d9-f9fe01ba7a97 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:55:56.107338 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc465b9-c6f8-4c28-88d9-f9fe01ba7a97 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:55:56.112091 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f42f4e-8248-475a-97a2-2d8bc5b1e0f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:55:56.112697 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f42f4e-8248-475a-97a2-2d8bc5b1e0f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:55:56.112971 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f42f4e-8248-475a-97a2-2d8bc5b1e0f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:55:56.113241 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f42f4e-8248-475a-97a2-2d8bc5b1e0f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:55:56.113885 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f42f4e-8248-475a-97a2-2d8bc5b1e0f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:55:56.115553 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f42f4e-8248-475a-97a2-2d8bc5b1e0f4 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:55:56.126328 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1254/2504] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:55:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:55:56.131012 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e823322-5f1d-42ef-b1f3-9b317c5e6343 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:55:56.141740 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e823322-5f1d-42ef-b1f3-9b317c5e6343 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:55:56.146591 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4426f274-bf26-4e42-84dd-972bb8474ad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:55:56.147771 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4426f274-bf26-4e42-84dd-972bb8474ad7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:55:56.148049 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4426f274-bf26-4e42-84dd-972bb8474ad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:55:56.148319 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4426f274-bf26-4e42-84dd-972bb8474ad7 admin admin] PATH_INFO: `/v3/projects/2d1b3b29c06a40f0b3405217c1389680/users/c0229c3a8fc54d5085e590642232f619/roles/74a40bc819cb42eea7e966a4b2a016f1` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:55:56.162865 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4426f274-bf26-4e42-84dd-972bb8474ad7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d1b3b29c06a40f0b3405217c1389680, user_id=c0229c3a8fc54d5085e590642232f619, role_id=74a40bc819cb42eea7e966a4b2a016f1)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:55:56.165559 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4426f274-bf26-4e42-84dd-972bb8474ad7 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:55:56.179709 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1251/2505] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:55:56 2022] PUT /identity/v3/projects/2d1b3b29c06a40f0b3405217c1389680/users/c0229c3a8fc54d5085e590642232f619/roles/74a40bc819cb42eea7e966a4b2a016f1 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:55:56.189860 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a59e81-231c-402f-80d8-b380b134617c None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:55:56.190145 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a59e81-231c-402f-80d8-b380b134617c None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:55:56.190429 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a59e81-231c-402f-80d8-b380b134617c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:55:56.220963 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-a2a59e81-231c-402f-80d8-b380b134617c None None] MFA Rules not processed for user `c0229c3a8fc54d5085e590642232f619`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:55:56.222177 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-a2a59e81-231c-402f-80d8-b380b134617c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:55:56.285876 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1255/2506] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:55:56 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:55:56.290429 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4426f274-bf26-4e42-84dd-972bb8474ad7 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:55:56.354016 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4426f274-bf26-4e42-84dd-972bb8474ad7 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:55:56.358643 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae595ca-1f9e-40d2-80d3-34e3e31a6b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:55:56.359243 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae595ca-1f9e-40d2-80d3-34e3e31a6b35 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:55:56.359516 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae595ca-1f9e-40d2-80d3-34e3e31a6b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:55:56.359821 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae595ca-1f9e-40d2-80d3-34e3e31a6b35 admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:55:56.360440 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae595ca-1f9e-40d2-80d3-34e3e31a6b35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:55:56.362315 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae595ca-1f9e-40d2-80d3-34e3e31a6b35 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:55:56.390639 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1252/2507] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 19:55:56 2022] POST /identity/v3/projects => generated 365 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:55:56.394789 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a59e81-231c-402f-80d8-b380b134617c None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:55:56.405935 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a59e81-231c-402f-80d8-b380b134617c None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:55:56.464785 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049c11f7-77c9-47d5-8255-6170b7b0eba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:55:56.465426 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049c11f7-77c9-47d5-8255-6170b7b0eba3 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:55:56.465705 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049c11f7-77c9-47d5-8255-6170b7b0eba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:55:56.465976 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049c11f7-77c9-47d5-8255-6170b7b0eba3 admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:55:56.466603 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049c11f7-77c9-47d5-8255-6170b7b0eba3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:55:56.468560 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049c11f7-77c9-47d5-8255-6170b7b0eba3 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:55:56.489120 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1256/2508] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 19:55:56 2022] POST /identity/v3/users => generated 289 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:55:56.493816 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae595ca-1f9e-40d2-80d3-34e3e31a6b35 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:55:56.500460 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:55:56.504414 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae595ca-1f9e-40d2-80d3-34e3e31a6b35 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:55:56.510941 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfd121c-cb4f-4ee9-836c-608fae0921e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:55:56.511743 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfd121c-cb4f-4ee9-836c-608fae0921e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:55:56.512237 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfd121c-cb4f-4ee9-836c-608fae0921e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:55:56.512559 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfd121c-cb4f-4ee9-836c-608fae0921e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:55:56.513375 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfd121c-cb4f-4ee9-836c-608fae0921e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:55:56.514985 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfd121c-cb4f-4ee9-836c-608fae0921e0 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:55:56.526990 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1253/2509] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:55:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:55:56.531440 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049c11f7-77c9-47d5-8255-6170b7b0eba3 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:55:56.542277 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049c11f7-77c9-47d5-8255-6170b7b0eba3 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:55:56.546827 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fa560f-0b9a-43f9-87f4-2bb504e95926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:55:56.547472 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fa560f-0b9a-43f9-87f4-2bb504e95926 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:55:56.547779 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fa560f-0b9a-43f9-87f4-2bb504e95926 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:55:56.548054 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fa560f-0b9a-43f9-87f4-2bb504e95926 admin admin] PATH_INFO: `/v3/projects/374ddd78a78e40a3a3a1a241bf1a26c1/users/944a2134e7b64931bb42969ce90319e5/roles/163c30c3b3c246beb849ce176f02d9da` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:55:56.563097 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fa560f-0b9a-43f9-87f4-2bb504e95926 admin admin] RBAC: Authorizing `identity:create_grant(project_id=374ddd78a78e40a3a3a1a241bf1a26c1, user_id=944a2134e7b64931bb42969ce90319e5, role_id=163c30c3b3c246beb849ce176f02d9da)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:55:56.565775 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fa560f-0b9a-43f9-87f4-2bb504e95926 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:55:56.578165 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1257/2510] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:55:56 2022] PUT /identity/v3/projects/374ddd78a78e40a3a3a1a241bf1a26c1/users/944a2134e7b64931bb42969ce90319e5/roles/163c30c3b3c246beb849ce176f02d9da => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:55:56.582056 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfd121c-cb4f-4ee9-836c-608fae0921e0 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:55:56.592412 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfd121c-cb4f-4ee9-836c-608fae0921e0 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:55:56.653903 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9a4d8e-0bf4-4739-9308-8b8c797d481b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:55:56.654510 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9a4d8e-0bf4-4739-9308-8b8c797d481b admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:55:56.654770 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9a4d8e-0bf4-4739-9308-8b8c797d481b admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:55:56.655036 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9a4d8e-0bf4-4739-9308-8b8c797d481b admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:55:56.655694 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9a4d8e-0bf4-4739-9308-8b8c797d481b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:55:56.657305 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9a4d8e-0bf4-4739-9308-8b8c797d481b admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:55:56.668120 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1254/2511] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:55:56 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 19:55:56.672239 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fa560f-0b9a-43f9-87f4-2bb504e95926 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:55:56.683227 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fa560f-0b9a-43f9-87f4-2bb504e95926 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:55:56.687895 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f65104-b43a-4c03-a6e7-9acd7ee21acc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:55:56.688550 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f65104-b43a-4c03-a6e7-9acd7ee21acc admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:55:56.688827 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f65104-b43a-4c03-a6e7-9acd7ee21acc admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:55:56.689100 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f65104-b43a-4c03-a6e7-9acd7ee21acc admin admin] PATH_INFO: `/v3/projects/374ddd78a78e40a3a3a1a241bf1a26c1/users/944a2134e7b64931bb42969ce90319e5/roles/163c30c3b3c246beb849ce176f02d9da` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:55:56.691039 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f65104-b43a-4c03-a6e7-9acd7ee21acc admin admin] RBAC: Authorizing `identity:create_grant(project_id=374ddd78a78e40a3a3a1a241bf1a26c1, user_id=944a2134e7b64931bb42969ce90319e5, role_id=163c30c3b3c246beb849ce176f02d9da)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:55:56.693686 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f65104-b43a-4c03-a6e7-9acd7ee21acc admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:55:56.699812 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:55:56.704751 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1258/2512] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:55:56 2022] PUT /identity/v3/projects/374ddd78a78e40a3a3a1a241bf1a26c1/users/944a2134e7b64931bb42969ce90319e5/roles/163c30c3b3c246beb849ce176f02d9da => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:55:56.716213 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d9ce08-ea21-445e-afc3-8b08bc19f65b None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:55:56.716576 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d9ce08-ea21-445e-afc3-8b08bc19f65b None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:55:56.716762 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d9ce08-ea21-445e-afc3-8b08bc19f65b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:55:56.746671 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-d8d9ce08-ea21-445e-afc3-8b08bc19f65b None None] MFA Rules not processed for user `944a2134e7b64931bb42969ce90319e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:55:56.747925 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-d8d9ce08-ea21-445e-afc3-8b08bc19f65b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:55:56.821513 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1255/2513] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:55:56 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:55:56.832786 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f65104-b43a-4c03-a6e7-9acd7ee21acc admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:55:56.834847 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-b8f65104-b43a-4c03-a6e7-9acd7ee21acc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:55:56.893212 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f65104-b43a-4c03-a6e7-9acd7ee21acc admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:55:56.897349 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12148642-7dda-4a01-a9c6-f6f13a55d5c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e79498873d4ed29a0e19bbea9b9214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:55:56.897953 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12148642-7dda-4a01-a9c6-f6f13a55d5c1 service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:55:56.898232 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12148642-7dda-4a01-a9c6-f6f13a55d5c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:55:56.898502 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12148642-7dda-4a01-a9c6-f6f13a55d5c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:55:56.907446 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12148642-7dda-4a01-a9c6-f6f13a55d5c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:55:56.908907 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12148642-7dda-4a01-a9c6-f6f13a55d5c1 service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:55:56.921397 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1259/2514] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:55:56 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:55:56.994583 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d9ce08-ea21-445e-afc3-8b08bc19f65b None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:55:57.004342 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d9ce08-ea21-445e-afc3-8b08bc19f65b None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:55:57.008018 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ecd820-544b-4c2f-bdf7-8a4cc0aa0095 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e79498873d4ed29a0e19bbea9b9214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:55:57.008618 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ecd820-544b-4c2f-bdf7-8a4cc0aa0095 service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:55:57.008891 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ecd820-544b-4c2f-bdf7-8a4cc0aa0095 service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:55:57.009186 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ecd820-544b-4c2f-bdf7-8a4cc0aa0095 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:55:57.066775 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ecd820-544b-4c2f-bdf7-8a4cc0aa0095 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:55:57.068268 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ecd820-544b-4c2f-bdf7-8a4cc0aa0095 service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:55:57.077211 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1256/2515] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:55:56 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:56:06.915220 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=26 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:56:07.073292 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=94 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:56:09.423926 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12148642-7dda-4a01-a9c6-f6f13a55d5c1 service cinder] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:56:09.487213 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12148642-7dda-4a01-a9c6-f6f13a55d5c1 service cinder] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:56:09.491750 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f0a770-551e-4ced-bd60-62903029662a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:56:09.492377 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f0a770-551e-4ced-bd60-62903029662a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:56:09.492653 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f0a770-551e-4ced-bd60-62903029662a admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:56:09.492943 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f0a770-551e-4ced-bd60-62903029662a admin admin] PATH_INFO: `/v3/users/c0229c3a8fc54d5085e590642232f619` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:56:09.494061 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f0a770-551e-4ced-bd60-62903029662a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0229c3a8fc54d5085e590642232f619)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:56:09.495995 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f0a770-551e-4ced-bd60-62903029662a admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:56:09.543523 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-f5f0a770-551e-4ced-bd60-62903029662a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0229c3a8fc54d5085e590642232f619'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:56:09.544125 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-f5f0a770-551e-4ced-bd60-62903029662a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0229c3a8fc54d5085e590642232f619'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:56:09.550920 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-f5f0a770-551e-4ced-bd60-62903029662a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0229c3a8fc54d5085e590642232f619'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:56:09.557905 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-f5f0a770-551e-4ced-bd60-62903029662a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0229c3a8fc54d5085e590642232f619'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:56:09.569669 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-f5f0a770-551e-4ced-bd60-62903029662a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0229c3a8fc54d5085e590642232f619'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:56:09.575987 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1260/2516] 10.222.0.61 () {58 vars in 1224 bytes} [Mon Aug 22 19:56:09 2022] DELETE /identity/v3/users/c0229c3a8fc54d5085e590642232f619 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:56:09.580455 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad9b579-8c98-4112-953f-eed7bb684f5f None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:56:09.580948 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad9b579-8c98-4112-953f-eed7bb684f5f None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:56:09.581041 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad9b579-8c98-4112-953f-eed7bb684f5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:56:09.600565 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-2ad9b579-8c98-4112-953f-eed7bb684f5f None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:56:09.601766 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-2ad9b579-8c98-4112-953f-eed7bb684f5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:56:09.659468 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1257/2517] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:56:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:56:09.669193 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f0a770-551e-4ced-bd60-62903029662a admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:56:09.670617 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-f5f0a770-551e-4ced-bd60-62903029662a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:56:09.729020 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f0a770-551e-4ced-bd60-62903029662a admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:56:09.732564 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa55a4b-f04d-431f-ba65-20fb24b37b51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:56:09.733165 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa55a4b-f04d-431f-ba65-20fb24b37b51 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:56:09.733440 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa55a4b-f04d-431f-ba65-20fb24b37b51 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:56:09.733886 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa55a4b-f04d-431f-ba65-20fb24b37b51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:56:09.743770 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa55a4b-f04d-431f-ba65-20fb24b37b51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:56:09.745213 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa55a4b-f04d-431f-ba65-20fb24b37b51 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:56:09.755683 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1261/2518] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:56:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:56:10.020667 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad9b579-8c98-4112-953f-eed7bb684f5f None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:56:10.022475 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-2ad9b579-8c98-4112-953f-eed7bb684f5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:56:10.042969 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad9b579-8c98-4112-953f-eed7bb684f5f None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:56:10.047977 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24f1a4f-d566-40a3-a646-21fae2ffd184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:56:10.048591 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24f1a4f-d566-40a3-a646-21fae2ffd184 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:56:10.049035 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24f1a4f-d566-40a3-a646-21fae2ffd184 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:56:10.049228 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24f1a4f-d566-40a3-a646-21fae2ffd184 admin admin] PATH_INFO: `/v3/projects/2d1b3b29c06a40f0b3405217c1389680` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:56:10.050410 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24f1a4f-d566-40a3-a646-21fae2ffd184 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d1b3b29c06a40f0b3405217c1389680)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:56:10.052510 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24f1a4f-d566-40a3-a646-21fae2ffd184 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:56:10.100413 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-c24f1a4f-d566-40a3-a646-21fae2ffd184 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d1b3b29c06a40f0b3405217c1389680', 'request_id': 'req-c24f1a4f-d566-40a3-a646-21fae2ffd184'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:56:10.108663 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-c24f1a4f-d566-40a3-a646-21fae2ffd184 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d1b3b29c06a40f0b3405217c1389680', 'request_id': 'req-c24f1a4f-d566-40a3-a646-21fae2ffd184'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:56:10.123314 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-c24f1a4f-d566-40a3-a646-21fae2ffd184 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d1b3b29c06a40f0b3405217c1389680', 'request_id': 'req-c24f1a4f-d566-40a3-a646-21fae2ffd184'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:56:10.134075 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-c24f1a4f-d566-40a3-a646-21fae2ffd184 admin admin] The token cache is being invalidate because project 2d1b3b29c06a40f0b3405217c1389680 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 19:56:10.134361 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-c24f1a4f-d566-40a3-a646-21fae2ffd184 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:56:10.135011 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-c24f1a4f-d566-40a3-a646-21fae2ffd184 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:56:10.136715 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1258/2519] 10.222.0.61 () {58 vars in 1233 bytes} [Mon Aug 22 19:56:10 2022] DELETE /identity/v3/projects/2d1b3b29c06a40f0b3405217c1389680 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:56:10.140804 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa55a4b-f04d-431f-ba65-20fb24b37b51 service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:56:10.150846 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa55a4b-f04d-431f-ba65-20fb24b37b51 service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:56:10.169015 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:56:10.209696 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235a1efa-8860-4fa9-9e93-078f62acfb89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:56:10.210324 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235a1efa-8860-4fa9-9e93-078f62acfb89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:56:10.210603 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235a1efa-8860-4fa9-9e93-078f62acfb89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:56:10.210875 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235a1efa-8860-4fa9-9e93-078f62acfb89 admin admin] PATH_INFO: `/v3/users/944a2134e7b64931bb42969ce90319e5` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:56:10.212038 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235a1efa-8860-4fa9-9e93-078f62acfb89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=944a2134e7b64931bb42969ce90319e5)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:56:10.213963 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235a1efa-8860-4fa9-9e93-078f62acfb89 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:56:10.260464 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-235a1efa-8860-4fa9-9e93-078f62acfb89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '944a2134e7b64931bb42969ce90319e5'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:56:10.261107 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-235a1efa-8860-4fa9-9e93-078f62acfb89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '944a2134e7b64931bb42969ce90319e5'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:56:10.268076 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-235a1efa-8860-4fa9-9e93-078f62acfb89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '944a2134e7b64931bb42969ce90319e5'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:56:10.275378 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-235a1efa-8860-4fa9-9e93-078f62acfb89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '944a2134e7b64931bb42969ce90319e5'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:56:10.287234 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-235a1efa-8860-4fa9-9e93-078f62acfb89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '944a2134e7b64931bb42969ce90319e5'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:56:10.293727 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1262/2520] 10.222.0.61 () {58 vars in 1224 bytes} [Mon Aug 22 19:56:10 2022] DELETE /identity/v3/users/944a2134e7b64931bb42969ce90319e5 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:56:10.565511 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24f1a4f-d566-40a3-a646-21fae2ffd184 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:56:10.567034 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-c24f1a4f-d566-40a3-a646-21fae2ffd184 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:56:10.580835 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24f1a4f-d566-40a3-a646-21fae2ffd184 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:56:10.638419 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdb1e6b-9733-4d8f-993a-acfa6ea3646d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:56:10.639047 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdb1e6b-9733-4d8f-993a-acfa6ea3646d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:56:10.639325 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdb1e6b-9733-4d8f-993a-acfa6ea3646d admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:56:10.639598 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdb1e6b-9733-4d8f-993a-acfa6ea3646d admin admin] PATH_INFO: `/v3/projects/374ddd78a78e40a3a3a1a241bf1a26c1` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:56:10.640737 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdb1e6b-9733-4d8f-993a-acfa6ea3646d admin admin] RBAC: Authorizing `identity:delete_project(project_id=374ddd78a78e40a3a3a1a241bf1a26c1)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:56:10.642697 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdb1e6b-9733-4d8f-993a-acfa6ea3646d admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:56:10.673768 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:56:10.691516 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-9bdb1e6b-9733-4d8f-993a-acfa6ea3646d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '374ddd78a78e40a3a3a1a241bf1a26c1', 'request_id': 'req-9bdb1e6b-9733-4d8f-993a-acfa6ea3646d'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:56:10.698777 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-9bdb1e6b-9733-4d8f-993a-acfa6ea3646d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '374ddd78a78e40a3a3a1a241bf1a26c1', 'request_id': 'req-9bdb1e6b-9733-4d8f-993a-acfa6ea3646d'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:56:10.704817 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-9bdb1e6b-9733-4d8f-993a-acfa6ea3646d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '374ddd78a78e40a3a3a1a241bf1a26c1', 'request_id': 'req-9bdb1e6b-9733-4d8f-993a-acfa6ea3646d'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:56:10.715897 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-9bdb1e6b-9733-4d8f-993a-acfa6ea3646d admin admin] The token cache is being invalidate because project 374ddd78a78e40a3a3a1a241bf1a26c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 19:56:10.716183 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-9bdb1e6b-9733-4d8f-993a-acfa6ea3646d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:56:10.716809 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-9bdb1e6b-9733-4d8f-993a-acfa6ea3646d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:56:10.718413 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1259/2521] 10.222.0.61 () {58 vars in 1233 bytes} [Mon Aug 22 19:56:10 2022] DELETE /identity/v3/projects/374ddd78a78e40a3a3a1a241bf1a26c1 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:56:10.723324 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546450b0-afb6-4176-8ee0-676097eb8f7f None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:56:10.723833 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546450b0-afb6-4176-8ee0-676097eb8f7f None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:56:10.724020 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546450b0-afb6-4176-8ee0-676097eb8f7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:56:10.744362 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-546450b0-afb6-4176-8ee0-676097eb8f7f None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:56:10.745608 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-546450b0-afb6-4176-8ee0-676097eb8f7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:56:10.806875 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1263/2522] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:56:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:56:10.814336 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc14c96-6319-4586-9f4c-108a55ab92d7 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:56:10.814648 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc14c96-6319-4586-9f4c-108a55ab92d7 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:56:10.814925 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc14c96-6319-4586-9f4c-108a55ab92d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:56:10.836843 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-6fc14c96-6319-4586-9f4c-108a55ab92d7 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:56:10.838542 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-6fc14c96-6319-4586-9f4c-108a55ab92d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:56:10.846379 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1260/2523] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:56:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:56:10.850507 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546450b0-afb6-4176-8ee0-676097eb8f7f None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:56:10.860372 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546450b0-afb6-4176-8ee0-676097eb8f7f None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:56:10.864986 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9ecb2a-d8d7-4630-b728-7294289a8ce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:56:10.865580 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9ecb2a-d8d7-4630-b728-7294289a8ce0 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:56:10.865860 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9ecb2a-d8d7-4630-b728-7294289a8ce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:56:10.866131 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9ecb2a-d8d7-4630-b728-7294289a8ce0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:56:10.866787 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9ecb2a-d8d7-4630-b728-7294289a8ce0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:56:10.867081 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9ecb2a-d8d7-4630-b728-7294289a8ce0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:56:10.868998 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9ecb2a-d8d7-4630-b728-7294289a8ce0 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:56:10.880566 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1264/2524] 10.222.0.61 () {58 vars in 1153 bytes} [Mon Aug 22 19:56:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:56:10.885277 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc14c96-6319-4586-9f4c-108a55ab92d7 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:56:10.894559 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc14c96-6319-4586-9f4c-108a55ab92d7 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:56:10.899059 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9eb482-a63d-43ab-9006-caad81070e89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:56:10.900283 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9eb482-a63d-43ab-9006-caad81070e89 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:56:10.900578 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9eb482-a63d-43ab-9006-caad81070e89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:56:10.900840 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9eb482-a63d-43ab-9006-caad81070e89 admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:56:10.901463 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9eb482-a63d-43ab-9006-caad81070e89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:56:10.903335 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9eb482-a63d-43ab-9006-caad81070e89 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:56:10.930691 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1261/2525] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 19:56:10 2022] POST /identity/v3/projects => generated 367 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:56:10.934914 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9ecb2a-d8d7-4630-b728-7294289a8ce0 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:56:10.944973 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9ecb2a-d8d7-4630-b728-7294289a8ce0 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:56:10.971525 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:56:11.002747 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0595157a-0500-4d32-a59b-b34454954efc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:56:11.003353 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0595157a-0500-4d32-a59b-b34454954efc admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:56:11.003637 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0595157a-0500-4d32-a59b-b34454954efc admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:56:11.003935 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0595157a-0500-4d32-a59b-b34454954efc admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:56:11.004561 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0595157a-0500-4d32-a59b-b34454954efc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:56:11.006478 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0595157a-0500-4d32-a59b-b34454954efc admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:56:11.028303 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1265/2526] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 19:56:10 2022] POST /identity/v3/users => generated 284 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 19:56:11.032519 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9eb482-a63d-43ab-9006-caad81070e89 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:56:11.042791 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9eb482-a63d-43ab-9006-caad81070e89 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:56:11.047383 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1594ea11-5c1d-4e6e-8764-d06b15a081de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:56:11.048011 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1594ea11-5c1d-4e6e-8764-d06b15a081de admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:56:11.048286 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1594ea11-5c1d-4e6e-8764-d06b15a081de admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:56:11.048555 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1594ea11-5c1d-4e6e-8764-d06b15a081de admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:56:11.049188 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1594ea11-5c1d-4e6e-8764-d06b15a081de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:56:11.050804 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1594ea11-5c1d-4e6e-8764-d06b15a081de admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:56:11.060737 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1262/2527] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:56:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:56:11.064508 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0595157a-0500-4d32-a59b-b34454954efc admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:56:11.074901 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0595157a-0500-4d32-a59b-b34454954efc admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:56:11.079507 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412c8606-c070-4b8d-a82c-25307bbeda0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:56:11.080131 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412c8606-c070-4b8d-a82c-25307bbeda0a admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:56:11.080410 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412c8606-c070-4b8d-a82c-25307bbeda0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:56:11.080682 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412c8606-c070-4b8d-a82c-25307bbeda0a admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:56:11.081325 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412c8606-c070-4b8d-a82c-25307bbeda0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:56:11.083169 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412c8606-c070-4b8d-a82c-25307bbeda0a admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:56:11.094168 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1266/2528] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:56:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:56:11.098273 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1594ea11-5c1d-4e6e-8764-d06b15a081de admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:56:11.108096 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1594ea11-5c1d-4e6e-8764-d06b15a081de admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:56:11.113522 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2817b386-20ac-4c5d-a003-fd1d3e3ca461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:56:11.114791 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2817b386-20ac-4c5d-a003-fd1d3e3ca461 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:56:11.115065 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2817b386-20ac-4c5d-a003-fd1d3e3ca461 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:56:11.115369 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2817b386-20ac-4c5d-a003-fd1d3e3ca461 admin admin] PATH_INFO: `/v3/projects/f567bfd7320b4518802e3d4ba6dc27f0/users/5c2445f18734408aa02bea4baad115ee/roles/74a40bc819cb42eea7e966a4b2a016f1` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:56:11.131339 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2817b386-20ac-4c5d-a003-fd1d3e3ca461 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f567bfd7320b4518802e3d4ba6dc27f0, user_id=5c2445f18734408aa02bea4baad115ee, role_id=74a40bc819cb42eea7e966a4b2a016f1)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:56:11.134650 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2817b386-20ac-4c5d-a003-fd1d3e3ca461 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:56:11.149260 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1263/2529] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:56:11 2022] PUT /identity/v3/projects/f567bfd7320b4518802e3d4ba6dc27f0/users/5c2445f18734408aa02bea4baad115ee/roles/74a40bc819cb42eea7e966a4b2a016f1 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:56:11.160689 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83165a33-7ef8-4756-af58-72d50c61a73c None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:56:11.160980 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83165a33-7ef8-4756-af58-72d50c61a73c None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:56:11.161246 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83165a33-7ef8-4756-af58-72d50c61a73c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:56:11.193067 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-83165a33-7ef8-4756-af58-72d50c61a73c None None] MFA Rules not processed for user `5c2445f18734408aa02bea4baad115ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:56:11.194266 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-83165a33-7ef8-4756-af58-72d50c61a73c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:56:11.257298 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1267/2530] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:56:11 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:56:11.262188 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2817b386-20ac-4c5d-a003-fd1d3e3ca461 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:56:11.323545 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2817b386-20ac-4c5d-a003-fd1d3e3ca461 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:56:11.328253 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f51298b-4341-4b25-bd47-689203afce63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:56:11.328848 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f51298b-4341-4b25-bd47-689203afce63 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:56:11.329126 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f51298b-4341-4b25-bd47-689203afce63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:56:11.329395 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f51298b-4341-4b25-bd47-689203afce63 admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:56:11.330026 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f51298b-4341-4b25-bd47-689203afce63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:56:11.331962 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f51298b-4341-4b25-bd47-689203afce63 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:56:11.362926 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1264/2531] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 19:56:11 2022] POST /identity/v3/projects => generated 369 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:56:11.364141 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83165a33-7ef8-4756-af58-72d50c61a73c None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:56:11.375623 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83165a33-7ef8-4756-af58-72d50c61a73c None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:56:11.433557 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e92c4d2-2eba-44eb-9460-d616b52f5d27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:56:11.434173 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e92c4d2-2eba-44eb-9460-d616b52f5d27 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:56:11.434453 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e92c4d2-2eba-44eb-9460-d616b52f5d27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:56:11.434727 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e92c4d2-2eba-44eb-9460-d616b52f5d27 admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:56:11.435352 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e92c4d2-2eba-44eb-9460-d616b52f5d27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:56:11.437295 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e92c4d2-2eba-44eb-9460-d616b52f5d27 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:56:11.459157 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1268/2532] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 19:56:11 2022] POST /identity/v3/users => generated 291 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:56:11.463492 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f51298b-4341-4b25-bd47-689203afce63 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:56:11.474068 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f51298b-4341-4b25-bd47-689203afce63 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:56:11.479408 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1004c9-deb1-4cad-abc8-ac8681b4ef1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:56:11.480154 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1004c9-deb1-4cad-abc8-ac8681b4ef1c admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:56:11.480431 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1004c9-deb1-4cad-abc8-ac8681b4ef1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:56:11.480698 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1004c9-deb1-4cad-abc8-ac8681b4ef1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:56:11.481335 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1004c9-deb1-4cad-abc8-ac8681b4ef1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:56:11.483019 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1004c9-deb1-4cad-abc8-ac8681b4ef1c admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:56:11.494584 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1265/2533] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:56:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:56:11.498511 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e92c4d2-2eba-44eb-9460-d616b52f5d27 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:56:11.508288 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e92c4d2-2eba-44eb-9460-d616b52f5d27 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:56:11.513131 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eae2cca-3fe4-49f7-bf21-a9f54ffe7a8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:56:11.513812 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eae2cca-3fe4-49f7-bf21-a9f54ffe7a8d admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:56:11.514095 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eae2cca-3fe4-49f7-bf21-a9f54ffe7a8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:56:11.514369 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eae2cca-3fe4-49f7-bf21-a9f54ffe7a8d admin admin] PATH_INFO: `/v3/projects/a3fc5c9e85014daea826a533431ecac9/users/97edb4e8f4f34f60bbb0da0bff03df93/roles/163c30c3b3c246beb849ce176f02d9da` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:56:11.529751 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eae2cca-3fe4-49f7-bf21-a9f54ffe7a8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3fc5c9e85014daea826a533431ecac9, user_id=97edb4e8f4f34f60bbb0da0bff03df93, role_id=163c30c3b3c246beb849ce176f02d9da)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:56:11.532499 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eae2cca-3fe4-49f7-bf21-a9f54ffe7a8d admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:56:11.546309 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1269/2534] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:56:11 2022] PUT /identity/v3/projects/a3fc5c9e85014daea826a533431ecac9/users/97edb4e8f4f34f60bbb0da0bff03df93/roles/163c30c3b3c246beb849ce176f02d9da => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:56:11.550247 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1004c9-deb1-4cad-abc8-ac8681b4ef1c admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:56:11.560864 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1004c9-deb1-4cad-abc8-ac8681b4ef1c admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:56:11.580669 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:56:11.619632 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc91134e-c26d-450a-a22c-9ff885fa7d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:56:11.620321 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc91134e-c26d-450a-a22c-9ff885fa7d98 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:56:11.620514 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc91134e-c26d-450a-a22c-9ff885fa7d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:56:11.620781 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc91134e-c26d-450a-a22c-9ff885fa7d98 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:56:11.621410 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc91134e-c26d-450a-a22c-9ff885fa7d98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:56:11.622983 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc91134e-c26d-450a-a22c-9ff885fa7d98 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:56:11.632743 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1266/2535] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:56:11 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:56:11.636886 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eae2cca-3fe4-49f7-bf21-a9f54ffe7a8d admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:56:11.642434 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:56:11.648479 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eae2cca-3fe4-49f7-bf21-a9f54ffe7a8d admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:56:11.653539 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7364bf89-b4cc-427a-b73c-cd17b7d1dc7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:56:11.654185 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7364bf89-b4cc-427a-b73c-cd17b7d1dc7f admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:56:11.654461 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7364bf89-b4cc-427a-b73c-cd17b7d1dc7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:56:11.654736 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7364bf89-b4cc-427a-b73c-cd17b7d1dc7f admin admin] PATH_INFO: `/v3/projects/a3fc5c9e85014daea826a533431ecac9/users/97edb4e8f4f34f60bbb0da0bff03df93/roles/163c30c3b3c246beb849ce176f02d9da` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:56:11.656958 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7364bf89-b4cc-427a-b73c-cd17b7d1dc7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3fc5c9e85014daea826a533431ecac9, user_id=97edb4e8f4f34f60bbb0da0bff03df93, role_id=163c30c3b3c246beb849ce176f02d9da)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:56:11.659569 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7364bf89-b4cc-427a-b73c-cd17b7d1dc7f admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:56:11.670942 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1270/2536] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:56:11 2022] PUT /identity/v3/projects/a3fc5c9e85014daea826a533431ecac9/users/97edb4e8f4f34f60bbb0da0bff03df93/roles/163c30c3b3c246beb849ce176f02d9da => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:56:11.681434 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d0ebd5-bc28-4975-8ce7-1213b80e668c None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:56:11.681712 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d0ebd5-bc28-4975-8ce7-1213b80e668c None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:56:11.681980 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d0ebd5-bc28-4975-8ce7-1213b80e668c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:56:11.708952 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-63d0ebd5-bc28-4975-8ce7-1213b80e668c None None] MFA Rules not processed for user `97edb4e8f4f34f60bbb0da0bff03df93`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:56:11.710064 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-63d0ebd5-bc28-4975-8ce7-1213b80e668c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:56:11.778318 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1267/2537] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:56:11 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:56:11.789564 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7364bf89-b4cc-427a-b73c-cd17b7d1dc7f admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:56:11.791872 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-7364bf89-b4cc-427a-b73c-cd17b7d1dc7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:56:11.850673 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7364bf89-b4cc-427a-b73c-cd17b7d1dc7f admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:56:11.854871 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21cf1d3-79e6-4da6-8b23-0eb925b5893c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e79498873d4ed29a0e19bbea9b9214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:56:11.855470 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21cf1d3-79e6-4da6-8b23-0eb925b5893c service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:56:11.855778 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21cf1d3-79e6-4da6-8b23-0eb925b5893c service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:56:11.856050 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21cf1d3-79e6-4da6-8b23-0eb925b5893c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:56:11.864968 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21cf1d3-79e6-4da6-8b23-0eb925b5893c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:56:11.866382 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21cf1d3-79e6-4da6-8b23-0eb925b5893c service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:56:11.876254 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1271/2538] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:56:11 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:56:11.920667 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d0ebd5-bc28-4975-8ce7-1213b80e668c None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:56:11.922131 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-63d0ebd5-bc28-4975-8ce7-1213b80e668c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:56:11.934634 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d0ebd5-bc28-4975-8ce7-1213b80e668c None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:56:11.938382 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934bcbe7-e0ec-4b56-aa12-7132e279cb77 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e79498873d4ed29a0e19bbea9b9214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:56:11.939016 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934bcbe7-e0ec-4b56-aa12-7132e279cb77 service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:56:11.939295 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934bcbe7-e0ec-4b56-aa12-7132e279cb77 service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:56:11.939597 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934bcbe7-e0ec-4b56-aa12-7132e279cb77 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:56:11.995550 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934bcbe7-e0ec-4b56-aa12-7132e279cb77 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:56:11.996916 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934bcbe7-e0ec-4b56-aa12-7132e279cb77 service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:56:12.005701 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1268/2539] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:56:11 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:56:15.272450 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21cf1d3-79e6-4da6-8b23-0eb925b5893c service cinder] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:56:15.332517 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21cf1d3-79e6-4da6-8b23-0eb925b5893c service cinder] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:56:15.337294 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d43a2a-c79a-4ba3-8bff-8b4254a85483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:56:15.337912 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d43a2a-c79a-4ba3-8bff-8b4254a85483 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:56:15.338187 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d43a2a-c79a-4ba3-8bff-8b4254a85483 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:56:15.338454 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d43a2a-c79a-4ba3-8bff-8b4254a85483 admin admin] PATH_INFO: `/v3/users/5c2445f18734408aa02bea4baad115ee` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:56:15.339566 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d43a2a-c79a-4ba3-8bff-8b4254a85483 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c2445f18734408aa02bea4baad115ee)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:56:15.341498 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d43a2a-c79a-4ba3-8bff-8b4254a85483 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:56:15.390741 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-e9d43a2a-c79a-4ba3-8bff-8b4254a85483 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c2445f18734408aa02bea4baad115ee'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:56:15.391395 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-e9d43a2a-c79a-4ba3-8bff-8b4254a85483 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c2445f18734408aa02bea4baad115ee'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:56:15.398818 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-e9d43a2a-c79a-4ba3-8bff-8b4254a85483 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c2445f18734408aa02bea4baad115ee'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:56:15.407194 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-e9d43a2a-c79a-4ba3-8bff-8b4254a85483 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c2445f18734408aa02bea4baad115ee'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:56:15.419147 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-e9d43a2a-c79a-4ba3-8bff-8b4254a85483 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c2445f18734408aa02bea4baad115ee'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:56:15.425162 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1272/2540] 10.222.0.61 () {58 vars in 1224 bytes} [Mon Aug 22 19:56:15 2022] DELETE /identity/v3/users/5c2445f18734408aa02bea4baad115ee => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:56:15.429608 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e1e81b-7136-40c1-b415-4e7f7466f34b None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:56:15.430126 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e1e81b-7136-40c1-b415-4e7f7466f34b None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:56:15.430126 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e1e81b-7136-40c1-b415-4e7f7466f34b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:56:15.449965 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-a6e1e81b-7136-40c1-b415-4e7f7466f34b None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:56:15.451320 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-a6e1e81b-7136-40c1-b415-4e7f7466f34b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:56:15.490932 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:56:15.509234 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1269/2541] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:56:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:56:15.519557 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d43a2a-c79a-4ba3-8bff-8b4254a85483 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:56:15.520950 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-e9d43a2a-c79a-4ba3-8bff-8b4254a85483 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:56:15.535148 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:56:15.581791 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d43a2a-c79a-4ba3-8bff-8b4254a85483 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:56:15.585588 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26db0e9d-344f-492d-b849-08000564c7e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:56:15.586194 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26db0e9d-344f-492d-b849-08000564c7e6 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:56:15.586466 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26db0e9d-344f-492d-b849-08000564c7e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:56:15.586742 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26db0e9d-344f-492d-b849-08000564c7e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:56:15.597250 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26db0e9d-344f-492d-b849-08000564c7e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:56:15.599005 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26db0e9d-344f-492d-b849-08000564c7e6 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:56:15.608780 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1273/2542] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:56:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:56:15.872977 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e1e81b-7136-40c1-b415-4e7f7466f34b None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:56:15.874566 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-a6e1e81b-7136-40c1-b415-4e7f7466f34b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:56:15.887597 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e1e81b-7136-40c1-b415-4e7f7466f34b None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:56:15.892325 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3524f44-aae4-4a2c-bd54-1876728fc9c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:56:15.892961 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3524f44-aae4-4a2c-bd54-1876728fc9c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:56:15.893242 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3524f44-aae4-4a2c-bd54-1876728fc9c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:56:15.893513 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3524f44-aae4-4a2c-bd54-1876728fc9c9 admin admin] PATH_INFO: `/v3/projects/f567bfd7320b4518802e3d4ba6dc27f0` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:56:15.894789 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3524f44-aae4-4a2c-bd54-1876728fc9c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f567bfd7320b4518802e3d4ba6dc27f0)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:56:15.896718 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3524f44-aae4-4a2c-bd54-1876728fc9c9 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:56:15.944473 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-d3524f44-aae4-4a2c-bd54-1876728fc9c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f567bfd7320b4518802e3d4ba6dc27f0', 'request_id': 'req-d3524f44-aae4-4a2c-bd54-1876728fc9c9'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:56:15.952450 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-d3524f44-aae4-4a2c-bd54-1876728fc9c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f567bfd7320b4518802e3d4ba6dc27f0', 'request_id': 'req-d3524f44-aae4-4a2c-bd54-1876728fc9c9'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:56:15.958933 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-d3524f44-aae4-4a2c-bd54-1876728fc9c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f567bfd7320b4518802e3d4ba6dc27f0', 'request_id': 'req-d3524f44-aae4-4a2c-bd54-1876728fc9c9'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:56:15.969710 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-d3524f44-aae4-4a2c-bd54-1876728fc9c9 admin admin] The token cache is being invalidate because project f567bfd7320b4518802e3d4ba6dc27f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 19:56:15.969994 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-d3524f44-aae4-4a2c-bd54-1876728fc9c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:56:15.970620 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-d3524f44-aae4-4a2c-bd54-1876728fc9c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:56:15.972139 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1270/2543] 10.222.0.61 () {58 vars in 1233 bytes} [Mon Aug 22 19:56:15 2022] DELETE /identity/v3/projects/f567bfd7320b4518802e3d4ba6dc27f0 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:56:15.976079 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26db0e9d-344f-492d-b849-08000564c7e6 service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:56:15.985602 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26db0e9d-344f-492d-b849-08000564c7e6 service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:56:16.042414 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbfc45d-4708-41a9-bbe9-5d8d4c02420a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:56:16.043047 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbfc45d-4708-41a9-bbe9-5d8d4c02420a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:56:16.043319 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbfc45d-4708-41a9-bbe9-5d8d4c02420a admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:56:16.043581 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbfc45d-4708-41a9-bbe9-5d8d4c02420a admin admin] PATH_INFO: `/v3/users/97edb4e8f4f34f60bbb0da0bff03df93` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:56:16.045320 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbfc45d-4708-41a9-bbe9-5d8d4c02420a admin admin] RBAC: Authorizing `identity:delete_user(user_id=97edb4e8f4f34f60bbb0da0bff03df93)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:56:16.047170 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbfc45d-4708-41a9-bbe9-5d8d4c02420a admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:56:16.092320 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-0bbfc45d-4708-41a9-bbe9-5d8d4c02420a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97edb4e8f4f34f60bbb0da0bff03df93'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:56:16.092966 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-0bbfc45d-4708-41a9-bbe9-5d8d4c02420a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97edb4e8f4f34f60bbb0da0bff03df93'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:56:16.099840 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-0bbfc45d-4708-41a9-bbe9-5d8d4c02420a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97edb4e8f4f34f60bbb0da0bff03df93'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:56:16.107842 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-0bbfc45d-4708-41a9-bbe9-5d8d4c02420a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97edb4e8f4f34f60bbb0da0bff03df93'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:56:16.120572 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-0bbfc45d-4708-41a9-bbe9-5d8d4c02420a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97edb4e8f4f34f60bbb0da0bff03df93'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:56:16.127900 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1274/2544] 10.222.0.61 () {58 vars in 1224 bytes} [Mon Aug 22 19:56:15 2022] DELETE /identity/v3/users/97edb4e8f4f34f60bbb0da0bff03df93 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:56:16.394120 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3524f44-aae4-4a2c-bd54-1876728fc9c9 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:56:16.396940 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-d3524f44-aae4-4a2c-bd54-1876728fc9c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:56:16.412439 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3524f44-aae4-4a2c-bd54-1876728fc9c9 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:56:16.484379 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bba2fc5-dc18-420a-8133-8e84471ecb0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:56:16.485132 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bba2fc5-dc18-420a-8133-8e84471ecb0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:56:16.485408 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bba2fc5-dc18-420a-8133-8e84471ecb0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:56:16.485678 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bba2fc5-dc18-420a-8133-8e84471ecb0f admin admin] PATH_INFO: `/v3/projects/a3fc5c9e85014daea826a533431ecac9` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:56:16.486786 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bba2fc5-dc18-420a-8133-8e84471ecb0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3fc5c9e85014daea826a533431ecac9)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:56:16.488714 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bba2fc5-dc18-420a-8133-8e84471ecb0f admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:56:16.535861 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-5bba2fc5-dc18-420a-8133-8e84471ecb0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3fc5c9e85014daea826a533431ecac9', 'request_id': 'req-5bba2fc5-dc18-420a-8133-8e84471ecb0f'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:56:16.545359 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-5bba2fc5-dc18-420a-8133-8e84471ecb0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3fc5c9e85014daea826a533431ecac9', 'request_id': 'req-5bba2fc5-dc18-420a-8133-8e84471ecb0f'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:56:16.557626 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-5bba2fc5-dc18-420a-8133-8e84471ecb0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3fc5c9e85014daea826a533431ecac9', 'request_id': 'req-5bba2fc5-dc18-420a-8133-8e84471ecb0f'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:56:16.564121 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:56:16.568937 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-5bba2fc5-dc18-420a-8133-8e84471ecb0f admin admin] The token cache is being invalidate because project a3fc5c9e85014daea826a533431ecac9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 19:56:16.569326 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-5bba2fc5-dc18-420a-8133-8e84471ecb0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:56:16.570443 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-5bba2fc5-dc18-420a-8133-8e84471ecb0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:56:16.572280 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1271/2545] 10.222.0.61 () {58 vars in 1233 bytes} [Mon Aug 22 19:56:16 2022] DELETE /identity/v3/projects/a3fc5c9e85014daea826a533431ecac9 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:56:16.577769 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d661c0f-4cb7-420e-b86f-91c478c1860c None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:56:16.578254 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d661c0f-4cb7-420e-b86f-91c478c1860c None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:56:16.578344 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d661c0f-4cb7-420e-b86f-91c478c1860c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:56:16.598310 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-6d661c0f-4cb7-420e-b86f-91c478c1860c None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:56:16.599506 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-6d661c0f-4cb7-420e-b86f-91c478c1860c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:56:16.629468 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:56:16.662916 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1275/2546] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:56:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:56:16.671137 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f2dfa4-b358-4cd5-a2d0-187efdb02cad None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:56:16.671428 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f2dfa4-b358-4cd5-a2d0-187efdb02cad None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:56:16.671739 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f2dfa4-b358-4cd5-a2d0-187efdb02cad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:56:16.691498 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-04f2dfa4-b358-4cd5-a2d0-187efdb02cad None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:56:16.692877 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-04f2dfa4-b358-4cd5-a2d0-187efdb02cad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:56:16.702783 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1272/2547] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:56:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:56:16.707214 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d661c0f-4cb7-420e-b86f-91c478c1860c None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:56:16.717833 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d661c0f-4cb7-420e-b86f-91c478c1860c None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:56:16.722893 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ac5b81-703f-479b-9c34-133dc826f8f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:56:16.723493 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ac5b81-703f-479b-9c34-133dc826f8f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:56:16.723807 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ac5b81-703f-479b-9c34-133dc826f8f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:56:16.724077 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ac5b81-703f-479b-9c34-133dc826f8f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:56:16.724722 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ac5b81-703f-479b-9c34-133dc826f8f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:56:16.725021 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ac5b81-703f-479b-9c34-133dc826f8f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:56:16.726605 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ac5b81-703f-479b-9c34-133dc826f8f4 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:56:16.737898 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1276/2548] 10.222.0.61 () {58 vars in 1153 bytes} [Mon Aug 22 19:56:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:56:16.742932 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f2dfa4-b358-4cd5-a2d0-187efdb02cad None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:56:16.752643 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f2dfa4-b358-4cd5-a2d0-187efdb02cad None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:56:16.758431 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690c56db-1f30-4e7d-b9c3-11d848dd1a4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:56:16.759070 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690c56db-1f30-4e7d-b9c3-11d848dd1a4a admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:56:16.759351 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690c56db-1f30-4e7d-b9c3-11d848dd1a4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:56:16.759627 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690c56db-1f30-4e7d-b9c3-11d848dd1a4a admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:56:16.760821 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690c56db-1f30-4e7d-b9c3-11d848dd1a4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:56:16.762739 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690c56db-1f30-4e7d-b9c3-11d848dd1a4a admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:56:16.790677 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1273/2549] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 19:56:16 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:56:16.794898 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ac5b81-703f-479b-9c34-133dc826f8f4 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:56:16.804859 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ac5b81-703f-479b-9c34-133dc826f8f4 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:56:16.860600 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b8f1bb-27af-42c8-a130-9d89fe4975f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:56:16.861132 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b8f1bb-27af-42c8-a130-9d89fe4975f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:56:16.861408 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b8f1bb-27af-42c8-a130-9d89fe4975f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:56:16.861676 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b8f1bb-27af-42c8-a130-9d89fe4975f5 admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:56:16.862296 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b8f1bb-27af-42c8-a130-9d89fe4975f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:56:16.864234 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b8f1bb-27af-42c8-a130-9d89fe4975f5 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:56:16.886213 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1277/2550] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 19:56:16 2022] POST /identity/v3/users => generated 292 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:56:16.890339 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690c56db-1f30-4e7d-b9c3-11d848dd1a4a admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:56:16.900757 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690c56db-1f30-4e7d-b9c3-11d848dd1a4a admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:56:16.906065 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875ebd3e-b7cf-427b-ab75-73a9a2421cae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:56:16.906658 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875ebd3e-b7cf-427b-ab75-73a9a2421cae admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:56:16.906945 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875ebd3e-b7cf-427b-ab75-73a9a2421cae admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:56:16.907203 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875ebd3e-b7cf-427b-ab75-73a9a2421cae admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:56:16.907901 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875ebd3e-b7cf-427b-ab75-73a9a2421cae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:56:16.909491 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875ebd3e-b7cf-427b-ab75-73a9a2421cae admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:56:16.920008 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1274/2551] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:56:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:56:16.923907 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b8f1bb-27af-42c8-a130-9d89fe4975f5 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:56:16.934436 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b8f1bb-27af-42c8-a130-9d89fe4975f5 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:56:16.939605 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef76496f-553c-45d2-9504-958a5ca6f1da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:56:16.940417 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef76496f-553c-45d2-9504-958a5ca6f1da admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:56:16.940763 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef76496f-553c-45d2-9504-958a5ca6f1da admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:56:16.941118 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef76496f-553c-45d2-9504-958a5ca6f1da admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:56:16.941808 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef76496f-553c-45d2-9504-958a5ca6f1da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:56:16.943395 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef76496f-553c-45d2-9504-958a5ca6f1da admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:56:16.953972 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1278/2552] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:56:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:56:16.958500 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875ebd3e-b7cf-427b-ab75-73a9a2421cae admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:56:16.968458 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875ebd3e-b7cf-427b-ab75-73a9a2421cae admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:56:16.973101 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6bad2b-7586-4fe1-b5ce-bacc24e38721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:56:16.973742 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6bad2b-7586-4fe1-b5ce-bacc24e38721 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:56:16.974023 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6bad2b-7586-4fe1-b5ce-bacc24e38721 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:56:16.974305 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6bad2b-7586-4fe1-b5ce-bacc24e38721 admin admin] PATH_INFO: `/v3/projects/85f293dfa2014ca4bca4957239ff94e6/users/b85cfd57247d4959b6704a75698591e0/roles/74a40bc819cb42eea7e966a4b2a016f1` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:56:16.989415 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6bad2b-7586-4fe1-b5ce-bacc24e38721 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85f293dfa2014ca4bca4957239ff94e6, user_id=b85cfd57247d4959b6704a75698591e0, role_id=74a40bc819cb42eea7e966a4b2a016f1)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:56:16.992111 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6bad2b-7586-4fe1-b5ce-bacc24e38721 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:56:17.005358 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1275/2553] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:56:16 2022] PUT /identity/v3/projects/85f293dfa2014ca4bca4957239ff94e6/users/b85cfd57247d4959b6704a75698591e0/roles/74a40bc819cb42eea7e966a4b2a016f1 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:56:17.016722 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caba6e97-ced3-4ebc-b237-c5e2fd3f77c9 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:56:17.017013 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caba6e97-ced3-4ebc-b237-c5e2fd3f77c9 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:56:17.017292 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caba6e97-ced3-4ebc-b237-c5e2fd3f77c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:56:17.046484 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-caba6e97-ced3-4ebc-b237-c5e2fd3f77c9 None None] MFA Rules not processed for user `b85cfd57247d4959b6704a75698591e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:56:17.047750 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-caba6e97-ced3-4ebc-b237-c5e2fd3f77c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:56:17.111328 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1279/2554] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:56:17 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:56:17.115885 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6bad2b-7586-4fe1-b5ce-bacc24e38721 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:56:17.179863 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6bad2b-7586-4fe1-b5ce-bacc24e38721 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:56:17.185173 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1457102e-4ba0-4055-952d-29374ae66049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:56:17.185771 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1457102e-4ba0-4055-952d-29374ae66049 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:56:17.186045 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1457102e-4ba0-4055-952d-29374ae66049 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:56:17.186318 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1457102e-4ba0-4055-952d-29374ae66049 admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:56:17.186933 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1457102e-4ba0-4055-952d-29374ae66049 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:56:17.188882 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1457102e-4ba0-4055-952d-29374ae66049 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:56:17.215228 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1276/2555] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 19:56:17 2022] POST /identity/v3/projects => generated 385 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:56:17.219507 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caba6e97-ced3-4ebc-b237-c5e2fd3f77c9 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:56:17.230989 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caba6e97-ced3-4ebc-b237-c5e2fd3f77c9 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:56:17.258086 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:56:17.289634 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a739a4d2-d5bc-4ab9-8168-6f11fb494dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:56:17.290237 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a739a4d2-d5bc-4ab9-8168-6f11fb494dc9 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:56:17.290513 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a739a4d2-d5bc-4ab9-8168-6f11fb494dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:56:17.290796 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a739a4d2-d5bc-4ab9-8168-6f11fb494dc9 admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:56:17.291437 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a739a4d2-d5bc-4ab9-8168-6f11fb494dc9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:56:17.293399 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a739a4d2-d5bc-4ab9-8168-6f11fb494dc9 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:56:17.314423 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1280/2556] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 19:56:17 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:56:17.318248 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1457102e-4ba0-4055-952d-29374ae66049 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:56:17.329113 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1457102e-4ba0-4055-952d-29374ae66049 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:56:17.333969 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6165227-5a57-4753-9640-110ce56cb6d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:56:17.334609 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6165227-5a57-4753-9640-110ce56cb6d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:56:17.334883 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6165227-5a57-4753-9640-110ce56cb6d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:56:17.335153 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6165227-5a57-4753-9640-110ce56cb6d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:56:17.335898 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6165227-5a57-4753-9640-110ce56cb6d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:56:17.337475 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6165227-5a57-4753-9640-110ce56cb6d5 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:56:17.348752 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1277/2557] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:56:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:56:17.352760 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a739a4d2-d5bc-4ab9-8168-6f11fb494dc9 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:56:17.362871 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a739a4d2-d5bc-4ab9-8168-6f11fb494dc9 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:56:17.367489 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dc4b22-246f-465d-987a-646df1954289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:56:17.368117 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dc4b22-246f-465d-987a-646df1954289 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:56:17.368397 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dc4b22-246f-465d-987a-646df1954289 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:56:17.368670 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dc4b22-246f-465d-987a-646df1954289 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:56:17.369314 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dc4b22-246f-465d-987a-646df1954289 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:56:17.371413 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dc4b22-246f-465d-987a-646df1954289 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:56:17.381350 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1281/2558] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:56:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:56:17.385483 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6165227-5a57-4753-9640-110ce56cb6d5 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:56:17.396230 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6165227-5a57-4753-9640-110ce56cb6d5 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:56:17.400785 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa41faa-4dd1-47f5-9f82-452200eae3f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:56:17.402166 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa41faa-4dd1-47f5-9f82-452200eae3f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:56:17.402563 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa41faa-4dd1-47f5-9f82-452200eae3f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:56:17.402839 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa41faa-4dd1-47f5-9f82-452200eae3f3 admin admin] PATH_INFO: `/v3/projects/fa538e130e764232a2ae59aeff95ba72/users/8e8ad9850afb44dba9ee6f7aef0a78ac/roles/74a40bc819cb42eea7e966a4b2a016f1` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:56:17.417866 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa41faa-4dd1-47f5-9f82-452200eae3f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa538e130e764232a2ae59aeff95ba72, user_id=8e8ad9850afb44dba9ee6f7aef0a78ac, role_id=74a40bc819cb42eea7e966a4b2a016f1)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:56:17.420535 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa41faa-4dd1-47f5-9f82-452200eae3f3 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:56:17.435952 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1278/2559] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:56:17 2022] PUT /identity/v3/projects/fa538e130e764232a2ae59aeff95ba72/users/8e8ad9850afb44dba9ee6f7aef0a78ac/roles/74a40bc819cb42eea7e966a4b2a016f1 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:56:17.446554 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f96eb58-9602-464c-8433-7b0d7bfb4837 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:56:17.446845 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f96eb58-9602-464c-8433-7b0d7bfb4837 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:56:17.447120 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f96eb58-9602-464c-8433-7b0d7bfb4837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:56:17.476134 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-3f96eb58-9602-464c-8433-7b0d7bfb4837 None None] MFA Rules not processed for user `8e8ad9850afb44dba9ee6f7aef0a78ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:56:17.477555 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-3f96eb58-9602-464c-8433-7b0d7bfb4837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:56:17.545071 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1282/2560] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:56:17 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:56:17.549800 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa41faa-4dd1-47f5-9f82-452200eae3f3 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:56:17.598162 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:56:17.615665 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa41faa-4dd1-47f5-9f82-452200eae3f3 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:56:17.620310 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a8e655-5cb2-4b71-8fb6-7899701b2373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:56:17.620910 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a8e655-5cb2-4b71-8fb6-7899701b2373 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:56:17.621187 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a8e655-5cb2-4b71-8fb6-7899701b2373 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:56:17.621455 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a8e655-5cb2-4b71-8fb6-7899701b2373 admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:56:17.622077 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a8e655-5cb2-4b71-8fb6-7899701b2373 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:56:17.624047 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a8e655-5cb2-4b71-8fb6-7899701b2373 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:56:17.650298 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1279/2561] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 19:56:17 2022] POST /identity/v3/projects => generated 387 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:56:17.654910 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f96eb58-9602-464c-8433-7b0d7bfb4837 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:56:17.667234 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f96eb58-9602-464c-8433-7b0d7bfb4837 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:56:17.727097 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b853fb-3c79-4595-8690-7e8c37ecec4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:56:17.727748 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b853fb-3c79-4595-8690-7e8c37ecec4f admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:56:17.728031 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b853fb-3c79-4595-8690-7e8c37ecec4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:56:17.728304 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b853fb-3c79-4595-8690-7e8c37ecec4f admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:56:17.728928 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b853fb-3c79-4595-8690-7e8c37ecec4f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:56:17.731007 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b853fb-3c79-4595-8690-7e8c37ecec4f admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:56:17.752541 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1283/2562] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 19:56:17 2022] POST /identity/v3/users => generated 300 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:56:17.757021 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a8e655-5cb2-4b71-8fb6-7899701b2373 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:56:17.768209 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a8e655-5cb2-4b71-8fb6-7899701b2373 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:56:17.772817 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21ac4a7-7bc2-4305-b64c-736762e2dc19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:56:17.773419 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21ac4a7-7bc2-4305-b64c-736762e2dc19 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:56:17.773701 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21ac4a7-7bc2-4305-b64c-736762e2dc19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:56:17.773973 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21ac4a7-7bc2-4305-b64c-736762e2dc19 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:56:17.774612 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21ac4a7-7bc2-4305-b64c-736762e2dc19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:56:17.776919 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21ac4a7-7bc2-4305-b64c-736762e2dc19 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:56:17.787430 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1280/2563] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:56:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 19:56:17.791695 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b853fb-3c79-4595-8690-7e8c37ecec4f admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:56:17.801770 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b853fb-3c79-4595-8690-7e8c37ecec4f admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:56:17.806356 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84f88dd-d1dd-4c85-819b-f58a076e4979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:56:17.806997 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84f88dd-d1dd-4c85-819b-f58a076e4979 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:56:17.807835 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84f88dd-d1dd-4c85-819b-f58a076e4979 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:56:17.808110 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84f88dd-d1dd-4c85-819b-f58a076e4979 admin admin] PATH_INFO: `/v3/projects/df737624e2414ced8b27f2e4c5157079/users/0fb8aa4b30064367b9437dc3acf6f326/roles/163c30c3b3c246beb849ce176f02d9da` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:56:17.822602 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84f88dd-d1dd-4c85-819b-f58a076e4979 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df737624e2414ced8b27f2e4c5157079, user_id=0fb8aa4b30064367b9437dc3acf6f326, role_id=163c30c3b3c246beb849ce176f02d9da)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:56:17.825907 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84f88dd-d1dd-4c85-819b-f58a076e4979 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:56:17.838789 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1284/2564] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:56:17 2022] PUT /identity/v3/projects/df737624e2414ced8b27f2e4c5157079/users/0fb8aa4b30064367b9437dc3acf6f326/roles/163c30c3b3c246beb849ce176f02d9da => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:56:17.842761 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21ac4a7-7bc2-4305-b64c-736762e2dc19 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:56:17.852810 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21ac4a7-7bc2-4305-b64c-736762e2dc19 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:56:17.910657 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33db01e9-c72a-4cf6-b861-5715ac4908f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:56:17.911172 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33db01e9-c72a-4cf6-b861-5715ac4908f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:56:17.911453 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33db01e9-c72a-4cf6-b861-5715ac4908f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:56:17.912616 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33db01e9-c72a-4cf6-b861-5715ac4908f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:56:17.912616 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33db01e9-c72a-4cf6-b861-5715ac4908f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:56:17.914073 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33db01e9-c72a-4cf6-b861-5715ac4908f7 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:56:17.925405 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1281/2565] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:56:17 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:56:17.929538 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84f88dd-d1dd-4c85-819b-f58a076e4979 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:56:17.935196 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:56:17.941777 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84f88dd-d1dd-4c85-819b-f58a076e4979 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:56:17.946910 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fab08e5-e6ca-4961-b36c-561242095ecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:56:17.947572 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fab08e5-e6ca-4961-b36c-561242095ecf admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:56:17.947881 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fab08e5-e6ca-4961-b36c-561242095ecf admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:56:17.948150 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fab08e5-e6ca-4961-b36c-561242095ecf admin admin] PATH_INFO: `/v3/projects/df737624e2414ced8b27f2e4c5157079/users/0fb8aa4b30064367b9437dc3acf6f326/roles/163c30c3b3c246beb849ce176f02d9da` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:56:17.950128 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fab08e5-e6ca-4961-b36c-561242095ecf admin admin] RBAC: Authorizing `identity:create_grant(project_id=df737624e2414ced8b27f2e4c5157079, user_id=0fb8aa4b30064367b9437dc3acf6f326, role_id=163c30c3b3c246beb849ce176f02d9da)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:56:17.952796 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fab08e5-e6ca-4961-b36c-561242095ecf admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:56:17.963125 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1285/2566] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:56:17 2022] PUT /identity/v3/projects/df737624e2414ced8b27f2e4c5157079/users/0fb8aa4b30064367b9437dc3acf6f326/roles/163c30c3b3c246beb849ce176f02d9da => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:56:17.974993 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69c5a4b-ca2e-4674-9337-580131630897 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:56:17.975307 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69c5a4b-ca2e-4674-9337-580131630897 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:56:17.975581 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69c5a4b-ca2e-4674-9337-580131630897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:56:18.004387 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-c69c5a4b-ca2e-4674-9337-580131630897 None None] MFA Rules not processed for user `0fb8aa4b30064367b9437dc3acf6f326`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:56:18.005645 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-c69c5a4b-ca2e-4674-9337-580131630897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:56:18.074294 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1282/2567] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:56:17 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:56:18.085353 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fab08e5-e6ca-4961-b36c-561242095ecf admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:56:18.087052 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-8fab08e5-e6ca-4961-b36c-561242095ecf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:56:18.146836 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fab08e5-e6ca-4961-b36c-561242095ecf admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:56:18.150100 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf3614b-b65c-4d9d-b0be-93bbd0685bc5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e79498873d4ed29a0e19bbea9b9214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:56:18.150702 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf3614b-b65c-4d9d-b0be-93bbd0685bc5 service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:56:18.151006 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf3614b-b65c-4d9d-b0be-93bbd0685bc5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:56:18.151275 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf3614b-b65c-4d9d-b0be-93bbd0685bc5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:56:18.160136 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf3614b-b65c-4d9d-b0be-93bbd0685bc5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:56:18.161598 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf3614b-b65c-4d9d-b0be-93bbd0685bc5 service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:56:18.171502 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1286/2568] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:56:18 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:56:18.684702 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69c5a4b-ca2e-4674-9337-580131630897 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:56:18.694422 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69c5a4b-ca2e-4674-9337-580131630897 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:56:18.698675 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385087f2-547d-4ba4-91f5-d9d232b1b450 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e79498873d4ed29a0e19bbea9b9214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:56:18.699271 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385087f2-547d-4ba4-91f5-d9d232b1b450 service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:56:18.699547 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385087f2-547d-4ba4-91f5-d9d232b1b450 service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:56:18.699846 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385087f2-547d-4ba4-91f5-d9d232b1b450 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:56:18.739399 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:56:18.762621 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385087f2-547d-4ba4-91f5-d9d232b1b450 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:56:18.764079 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385087f2-547d-4ba4-91f5-d9d232b1b450 service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:56:18.773038 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1283/2569] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:56:18 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:56:22.790430 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf3614b-b65c-4d9d-b0be-93bbd0685bc5 service cinder] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:56:22.792544 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-2cf3614b-b65c-4d9d-b0be-93bbd0685bc5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:56:22.803863 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf3614b-b65c-4d9d-b0be-93bbd0685bc5 service cinder] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:56:22.807437 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2834b7b1-dbb2-474e-8d85-cbcf737d6896 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e79498873d4ed29a0e19bbea9b9214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:56:22.808737 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2834b7b1-dbb2-474e-8d85-cbcf737d6896 service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:56:22.809014 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2834b7b1-dbb2-474e-8d85-cbcf737d6896 service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:56:22.809295 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2834b7b1-dbb2-474e-8d85-cbcf737d6896 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:56:22.882005 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2834b7b1-dbb2-474e-8d85-cbcf737d6896 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:56:22.883342 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2834b7b1-dbb2-474e-8d85-cbcf737d6896 service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:56:22.892180 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1287/2570] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:56:22 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:56:24.977898 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385087f2-547d-4ba4-91f5-d9d232b1b450 service cinder] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:56:25.040696 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385087f2-547d-4ba4-91f5-d9d232b1b450 service cinder] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:56:25.046127 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504c2e02-137a-4d29-b5fb-6a17af773806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:56:25.046783 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504c2e02-137a-4d29-b5fb-6a17af773806 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:56:25.047096 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504c2e02-137a-4d29-b5fb-6a17af773806 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:56:25.047368 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504c2e02-137a-4d29-b5fb-6a17af773806 admin admin] PATH_INFO: `/v3/users/b85cfd57247d4959b6704a75698591e0` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:56:25.048625 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504c2e02-137a-4d29-b5fb-6a17af773806 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b85cfd57247d4959b6704a75698591e0)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:56:25.050802 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504c2e02-137a-4d29-b5fb-6a17af773806 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:56:25.099853 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-504c2e02-137a-4d29-b5fb-6a17af773806 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b85cfd57247d4959b6704a75698591e0'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:56:25.101039 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-504c2e02-137a-4d29-b5fb-6a17af773806 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b85cfd57247d4959b6704a75698591e0'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:56:25.108772 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-504c2e02-137a-4d29-b5fb-6a17af773806 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b85cfd57247d4959b6704a75698591e0'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:56:25.117371 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-504c2e02-137a-4d29-b5fb-6a17af773806 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b85cfd57247d4959b6704a75698591e0'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:56:25.130202 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-504c2e02-137a-4d29-b5fb-6a17af773806 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b85cfd57247d4959b6704a75698591e0'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:56:25.136568 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1284/2571] 10.222.0.61 () {58 vars in 1224 bytes} [Mon Aug 22 19:56:24 2022] DELETE /identity/v3/users/b85cfd57247d4959b6704a75698591e0 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:56:25.141149 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e32162a-d8fe-4453-931a-e2f056bc3c01 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:56:25.141610 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e32162a-d8fe-4453-931a-e2f056bc3c01 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:56:25.141695 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e32162a-d8fe-4453-931a-e2f056bc3c01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:56:25.160954 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-0e32162a-d8fe-4453-931a-e2f056bc3c01 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:56:25.162205 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-0e32162a-d8fe-4453-931a-e2f056bc3c01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:56:25.221271 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1288/2572] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:56:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:56:25.231175 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504c2e02-137a-4d29-b5fb-6a17af773806 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:56:25.232619 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-504c2e02-137a-4d29-b5fb-6a17af773806 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:56:25.290624 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504c2e02-137a-4d29-b5fb-6a17af773806 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:56:25.294320 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e49001-728f-48cb-bef0-3ff31185e5c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:56:25.294923 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e49001-728f-48cb-bef0-3ff31185e5c2 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:56:25.295200 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e49001-728f-48cb-bef0-3ff31185e5c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:56:25.295465 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e49001-728f-48cb-bef0-3ff31185e5c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:56:25.304651 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e49001-728f-48cb-bef0-3ff31185e5c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:56:25.306082 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e49001-728f-48cb-bef0-3ff31185e5c2 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:56:25.311843 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:56:25.317484 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1285/2573] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:56:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:56:25.597514 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e32162a-d8fe-4453-931a-e2f056bc3c01 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:56:25.599093 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-0e32162a-d8fe-4453-931a-e2f056bc3c01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:56:25.612072 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e32162a-d8fe-4453-931a-e2f056bc3c01 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:56:25.617078 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b6e71d-c716-4e62-b55a-a600bfc1b33c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:56:25.617724 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b6e71d-c716-4e62-b55a-a600bfc1b33c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:56:25.618007 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b6e71d-c716-4e62-b55a-a600bfc1b33c admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:56:25.618280 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b6e71d-c716-4e62-b55a-a600bfc1b33c admin admin] PATH_INFO: `/v3/projects/85f293dfa2014ca4bca4957239ff94e6` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:56:25.619496 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b6e71d-c716-4e62-b55a-a600bfc1b33c admin admin] RBAC: Authorizing `identity:delete_project(project_id=85f293dfa2014ca4bca4957239ff94e6)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:56:25.621457 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b6e71d-c716-4e62-b55a-a600bfc1b33c admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:56:25.653045 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:56:25.671260 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-27b6e71d-c716-4e62-b55a-a600bfc1b33c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85f293dfa2014ca4bca4957239ff94e6', 'request_id': 'req-27b6e71d-c716-4e62-b55a-a600bfc1b33c'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:56:25.678731 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-27b6e71d-c716-4e62-b55a-a600bfc1b33c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85f293dfa2014ca4bca4957239ff94e6', 'request_id': 'req-27b6e71d-c716-4e62-b55a-a600bfc1b33c'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:56:25.686561 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-27b6e71d-c716-4e62-b55a-a600bfc1b33c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85f293dfa2014ca4bca4957239ff94e6', 'request_id': 'req-27b6e71d-c716-4e62-b55a-a600bfc1b33c'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:56:25.696803 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-27b6e71d-c716-4e62-b55a-a600bfc1b33c admin admin] The token cache is being invalidate because project 85f293dfa2014ca4bca4957239ff94e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 19:56:25.697089 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-27b6e71d-c716-4e62-b55a-a600bfc1b33c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:56:25.697731 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-27b6e71d-c716-4e62-b55a-a600bfc1b33c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:56:25.699231 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1289/2574] 10.222.0.61 () {58 vars in 1233 bytes} [Mon Aug 22 19:56:25 2022] DELETE /identity/v3/projects/85f293dfa2014ca4bca4957239ff94e6 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:56:25.703210 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e49001-728f-48cb-bef0-3ff31185e5c2 service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:56:25.713024 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e49001-728f-48cb-bef0-3ff31185e5c2 service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:56:25.770268 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9759d0-c51d-4130-bc5f-22c759e341ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:56:25.771096 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9759d0-c51d-4130-bc5f-22c759e341ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:56:25.771552 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9759d0-c51d-4130-bc5f-22c759e341ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:56:25.771878 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9759d0-c51d-4130-bc5f-22c759e341ae admin admin] PATH_INFO: `/v3/users/8e8ad9850afb44dba9ee6f7aef0a78ac` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:56:25.772998 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9759d0-c51d-4130-bc5f-22c759e341ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e8ad9850afb44dba9ee6f7aef0a78ac)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:56:25.774954 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9759d0-c51d-4130-bc5f-22c759e341ae admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:56:25.822914 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-fb9759d0-c51d-4130-bc5f-22c759e341ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e8ad9850afb44dba9ee6f7aef0a78ac'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:56:25.823551 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-fb9759d0-c51d-4130-bc5f-22c759e341ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e8ad9850afb44dba9ee6f7aef0a78ac'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:56:25.830494 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-fb9759d0-c51d-4130-bc5f-22c759e341ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e8ad9850afb44dba9ee6f7aef0a78ac'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:56:25.838189 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-fb9759d0-c51d-4130-bc5f-22c759e341ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e8ad9850afb44dba9ee6f7aef0a78ac'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:56:25.850705 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-fb9759d0-c51d-4130-bc5f-22c759e341ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e8ad9850afb44dba9ee6f7aef0a78ac'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:56:25.857375 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1286/2575] 10.222.0.61 () {58 vars in 1224 bytes} [Mon Aug 22 19:56:25 2022] DELETE /identity/v3/users/8e8ad9850afb44dba9ee6f7aef0a78ac => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 19:56:26.149487 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b6e71d-c716-4e62-b55a-a600bfc1b33c admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:56:26.149487 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-27b6e71d-c716-4e62-b55a-a600bfc1b33c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:56:26.170913 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b6e71d-c716-4e62-b55a-a600bfc1b33c admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:56:26.232765 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de07673b-0346-4a9e-b103-0e1eccef596d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:56:26.233400 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de07673b-0346-4a9e-b103-0e1eccef596d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:56:26.233681 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de07673b-0346-4a9e-b103-0e1eccef596d admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:56:26.233951 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de07673b-0346-4a9e-b103-0e1eccef596d admin admin] PATH_INFO: `/v3/projects/fa538e130e764232a2ae59aeff95ba72` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:56:26.235065 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de07673b-0346-4a9e-b103-0e1eccef596d admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa538e130e764232a2ae59aeff95ba72)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:56:26.237004 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de07673b-0346-4a9e-b103-0e1eccef596d admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:56:26.284552 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-de07673b-0346-4a9e-b103-0e1eccef596d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa538e130e764232a2ae59aeff95ba72', 'request_id': 'req-de07673b-0346-4a9e-b103-0e1eccef596d'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:56:26.291776 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-de07673b-0346-4a9e-b103-0e1eccef596d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa538e130e764232a2ae59aeff95ba72', 'request_id': 'req-de07673b-0346-4a9e-b103-0e1eccef596d'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:56:26.298507 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-de07673b-0346-4a9e-b103-0e1eccef596d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa538e130e764232a2ae59aeff95ba72', 'request_id': 'req-de07673b-0346-4a9e-b103-0e1eccef596d'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:56:26.308954 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-de07673b-0346-4a9e-b103-0e1eccef596d admin admin] The token cache is being invalidate because project fa538e130e764232a2ae59aeff95ba72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 19:56:26.309265 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-de07673b-0346-4a9e-b103-0e1eccef596d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:56:26.309905 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-de07673b-0346-4a9e-b103-0e1eccef596d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:56:26.311379 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1290/2576] 10.222.0.61 () {58 vars in 1233 bytes} [Mon Aug 22 19:56:26 2022] DELETE /identity/v3/projects/fa538e130e764232a2ae59aeff95ba72 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:56:26.315373 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9759d0-c51d-4130-bc5f-22c759e341ae admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:56:26.316887 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-fb9759d0-c51d-4130-bc5f-22c759e341ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:56:26.330183 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9759d0-c51d-4130-bc5f-22c759e341ae admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:56:26.390045 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89034ee-48b7-42f0-99a7-9c9e80055381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:56:26.390714 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89034ee-48b7-42f0-99a7-9c9e80055381 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:56:26.390993 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89034ee-48b7-42f0-99a7-9c9e80055381 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:56:26.391266 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89034ee-48b7-42f0-99a7-9c9e80055381 admin admin] PATH_INFO: `/v3/users/0fb8aa4b30064367b9437dc3acf6f326` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:56:26.392422 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89034ee-48b7-42f0-99a7-9c9e80055381 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fb8aa4b30064367b9437dc3acf6f326)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:56:26.394337 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89034ee-48b7-42f0-99a7-9c9e80055381 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:56:26.415583 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:56:26.443880 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-e89034ee-48b7-42f0-99a7-9c9e80055381 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fb8aa4b30064367b9437dc3acf6f326'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:56:26.444550 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-e89034ee-48b7-42f0-99a7-9c9e80055381 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fb8aa4b30064367b9437dc3acf6f326'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:56:26.452104 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-e89034ee-48b7-42f0-99a7-9c9e80055381 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fb8aa4b30064367b9437dc3acf6f326'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:56:26.460001 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-e89034ee-48b7-42f0-99a7-9c9e80055381 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fb8aa4b30064367b9437dc3acf6f326'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:56:26.472303 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-e89034ee-48b7-42f0-99a7-9c9e80055381 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fb8aa4b30064367b9437dc3acf6f326'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:56:26.478308 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1287/2577] 10.222.0.61 () {58 vars in 1224 bytes} [Mon Aug 22 19:56:26 2022] DELETE /identity/v3/users/0fb8aa4b30064367b9437dc3acf6f326 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:56:26.758091 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de07673b-0346-4a9e-b103-0e1eccef596d admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:56:26.759940 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-de07673b-0346-4a9e-b103-0e1eccef596d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:56:26.774681 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de07673b-0346-4a9e-b103-0e1eccef596d admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:56:26.821419 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:56:26.838765 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218c3f51-8e6f-479b-9164-4a5f46d292fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:56:26.839397 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218c3f51-8e6f-479b-9164-4a5f46d292fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:56:26.839697 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218c3f51-8e6f-479b-9164-4a5f46d292fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:56:26.839968 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218c3f51-8e6f-479b-9164-4a5f46d292fc admin admin] PATH_INFO: `/v3/projects/df737624e2414ced8b27f2e4c5157079` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:56:26.841097 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218c3f51-8e6f-479b-9164-4a5f46d292fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=df737624e2414ced8b27f2e4c5157079)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:56:26.842988 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218c3f51-8e6f-479b-9164-4a5f46d292fc admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:56:26.888820 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-218c3f51-8e6f-479b-9164-4a5f46d292fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df737624e2414ced8b27f2e4c5157079', 'request_id': 'req-218c3f51-8e6f-479b-9164-4a5f46d292fc'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:56:26.896378 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-218c3f51-8e6f-479b-9164-4a5f46d292fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df737624e2414ced8b27f2e4c5157079', 'request_id': 'req-218c3f51-8e6f-479b-9164-4a5f46d292fc'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:56:26.902902 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-218c3f51-8e6f-479b-9164-4a5f46d292fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df737624e2414ced8b27f2e4c5157079', 'request_id': 'req-218c3f51-8e6f-479b-9164-4a5f46d292fc'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:56:26.913150 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-218c3f51-8e6f-479b-9164-4a5f46d292fc admin admin] The token cache is being invalidate because project df737624e2414ced8b27f2e4c5157079 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 19:56:26.913435 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-218c3f51-8e6f-479b-9164-4a5f46d292fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:56:26.914064 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-218c3f51-8e6f-479b-9164-4a5f46d292fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:56:26.915540 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1291/2578] 10.222.0.61 () {58 vars in 1233 bytes} [Mon Aug 22 19:56:26 2022] DELETE /identity/v3/projects/df737624e2414ced8b27f2e4c5157079 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:56:26.922529 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f17035-07ae-4ee2-af9d-8f37d616c09d None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:56:26.922818 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f17035-07ae-4ee2-af9d-8f37d616c09d None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:56:26.923101 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f17035-07ae-4ee2-af9d-8f37d616c09d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:56:26.943579 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-85f17035-07ae-4ee2-af9d-8f37d616c09d None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:56:26.944815 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-85f17035-07ae-4ee2-af9d-8f37d616c09d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:56:27.007161 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1288/2579] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:56:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:56:27.015611 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578c6d42-de06-47a9-9047-a2236da97565 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:56:27.016005 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578c6d42-de06-47a9-9047-a2236da97565 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:56:27.016396 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578c6d42-de06-47a9-9047-a2236da97565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:56:27.037232 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-578c6d42-de06-47a9-9047-a2236da97565 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:56:27.038507 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-578c6d42-de06-47a9-9047-a2236da97565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:56:27.047268 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1292/2580] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:56:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:56:27.051892 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f17035-07ae-4ee2-af9d-8f37d616c09d None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:56:27.063055 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f17035-07ae-4ee2-af9d-8f37d616c09d None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:56:27.067666 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91bd949-4fee-459d-b324-584269bd66da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:56:27.068271 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91bd949-4fee-459d-b324-584269bd66da admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:56:27.068549 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91bd949-4fee-459d-b324-584269bd66da admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:56:27.068817 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91bd949-4fee-459d-b324-584269bd66da admin admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:56:27.069464 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91bd949-4fee-459d-b324-584269bd66da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:56:27.069763 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91bd949-4fee-459d-b324-584269bd66da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:56:27.071387 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91bd949-4fee-459d-b324-584269bd66da admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:56:27.083234 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1289/2581] 10.222.0.61 () {58 vars in 1153 bytes} [Mon Aug 22 19:56:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 19:56:27.087743 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578c6d42-de06-47a9-9047-a2236da97565 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:56:27.097395 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578c6d42-de06-47a9-9047-a2236da97565 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:56:27.102265 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c724d942-13bc-4bb5-881a-2cd6dcf6f13f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:56:27.102867 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c724d942-13bc-4bb5-881a-2cd6dcf6f13f admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:56:27.103144 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c724d942-13bc-4bb5-881a-2cd6dcf6f13f admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:56:27.103419 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c724d942-13bc-4bb5-881a-2cd6dcf6f13f admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:56:27.104084 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c724d942-13bc-4bb5-881a-2cd6dcf6f13f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:56:27.105979 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c724d942-13bc-4bb5-881a-2cd6dcf6f13f admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:56:27.136894 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1293/2582] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 19:56:27 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:56:27.142301 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91bd949-4fee-459d-b324-584269bd66da admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:56:27.153856 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91bd949-4fee-459d-b324-584269bd66da admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:56:27.211806 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cd0d0f-3c6d-426f-830b-c21a318c0d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:56:27.212331 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cd0d0f-3c6d-426f-830b-c21a318c0d0e admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:56:27.212607 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cd0d0f-3c6d-426f-830b-c21a318c0d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:56:27.212875 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cd0d0f-3c6d-426f-830b-c21a318c0d0e admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:56:27.213494 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cd0d0f-3c6d-426f-830b-c21a318c0d0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:56:27.215378 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cd0d0f-3c6d-426f-830b-c21a318c0d0e admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:56:27.237757 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1290/2583] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 19:56:27 2022] POST /identity/v3/users => generated 287 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:56:27.241819 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c724d942-13bc-4bb5-881a-2cd6dcf6f13f admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:56:27.252006 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c724d942-13bc-4bb5-881a-2cd6dcf6f13f admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:56:27.256769 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b253920-2ca1-4c5f-b5c6-00eb868f339a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:56:27.257383 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b253920-2ca1-4c5f-b5c6-00eb868f339a admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:56:27.257660 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b253920-2ca1-4c5f-b5c6-00eb868f339a admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:56:27.257926 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b253920-2ca1-4c5f-b5c6-00eb868f339a admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:56:27.258562 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b253920-2ca1-4c5f-b5c6-00eb868f339a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:56:27.260324 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b253920-2ca1-4c5f-b5c6-00eb868f339a admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:56:27.270056 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1294/2584] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:56:27 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:56:27.274145 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cd0d0f-3c6d-426f-830b-c21a318c0d0e admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:56:27.284963 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cd0d0f-3c6d-426f-830b-c21a318c0d0e admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:56:27.289756 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf21ebb8-68af-40a9-8ae1-f404306aed6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:56:27.290348 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf21ebb8-68af-40a9-8ae1-f404306aed6a admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:56:27.290625 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf21ebb8-68af-40a9-8ae1-f404306aed6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:56:27.290917 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf21ebb8-68af-40a9-8ae1-f404306aed6a admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:56:27.291541 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf21ebb8-68af-40a9-8ae1-f404306aed6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:56:27.293155 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf21ebb8-68af-40a9-8ae1-f404306aed6a admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:56:27.304836 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1291/2585] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:56:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:56:27.308815 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b253920-2ca1-4c5f-b5c6-00eb868f339a admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:56:27.319448 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b253920-2ca1-4c5f-b5c6-00eb868f339a admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:56:27.324494 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f050b2-360f-43dd-8c4c-3bc98b536966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:56:27.325195 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f050b2-360f-43dd-8c4c-3bc98b536966 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:56:27.325501 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f050b2-360f-43dd-8c4c-3bc98b536966 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:56:27.325798 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f050b2-360f-43dd-8c4c-3bc98b536966 admin admin] PATH_INFO: `/v3/projects/532a20c016764748a97e11413dab4aba/users/8c727c852d63447d89179e17d4731628/roles/74a40bc819cb42eea7e966a4b2a016f1` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:56:27.341306 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f050b2-360f-43dd-8c4c-3bc98b536966 admin admin] RBAC: Authorizing `identity:create_grant(project_id=532a20c016764748a97e11413dab4aba, user_id=8c727c852d63447d89179e17d4731628, role_id=74a40bc819cb42eea7e966a4b2a016f1)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:56:27.343962 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f050b2-360f-43dd-8c4c-3bc98b536966 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:56:27.357976 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1295/2586] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:56:27 2022] PUT /identity/v3/projects/532a20c016764748a97e11413dab4aba/users/8c727c852d63447d89179e17d4731628/roles/74a40bc819cb42eea7e966a4b2a016f1 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:56:27.363934 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92a1995-f920-412e-9685-d7e73992a862 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:56:27.364410 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92a1995-f920-412e-9685-d7e73992a862 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:56:27.364541 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92a1995-f920-412e-9685-d7e73992a862 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:56:27.372653 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:56:27.384202 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-f92a1995-f920-412e-9685-d7e73992a862 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:56:27.385395 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-f92a1995-f920-412e-9685-d7e73992a862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:56:27.444419 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1292/2587] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:56:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:56:27.454090 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f050b2-360f-43dd-8c4c-3bc98b536966 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:56:27.455881 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-83f050b2-360f-43dd-8c4c-3bc98b536966 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:56:27.515827 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f050b2-360f-43dd-8c4c-3bc98b536966 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:56:27.520060 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e95f31a-c5b7-49f6-9901-347d1cac116f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:56:27.520705 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e95f31a-c5b7-49f6-9901-347d1cac116f service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:56:27.520978 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e95f31a-c5b7-49f6-9901-347d1cac116f service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:56:27.521244 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e95f31a-c5b7-49f6-9901-347d1cac116f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:56:27.532647 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e95f31a-c5b7-49f6-9901-347d1cac116f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:56:27.534072 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e95f31a-c5b7-49f6-9901-347d1cac116f service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:56:27.543631 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1296/2588] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:56:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:56:28.158088 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fac016e-a2ed-4747-9e9a-16a4b3e9d1c1 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:56:28.158526 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fac016e-a2ed-4747-9e9a-16a4b3e9d1c1 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:56:28.160037 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fac016e-a2ed-4747-9e9a-16a4b3e9d1c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:56:28.186749 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-2fac016e-a2ed-4747-9e9a-16a4b3e9d1c1 None None] MFA Rules not processed for user `8c727c852d63447d89179e17d4731628`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:56:28.188009 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-2fac016e-a2ed-4747-9e9a-16a4b3e9d1c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:56:28.267555 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1293/2589] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:56:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:56:28.280050 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e95f31a-c5b7-49f6-9901-347d1cac116f service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:56:28.281550 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-9e95f31a-c5b7-49f6-9901-347d1cac116f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:56:28.296710 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:56:28.344820 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e95f31a-c5b7-49f6-9901-347d1cac116f service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:56:28.348438 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fe6563-7235-4c06-ab52-3933c2e57f5d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e79498873d4ed29a0e19bbea9b9214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:56:28.349048 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fe6563-7235-4c06-ab52-3933c2e57f5d service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:56:28.349355 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fe6563-7235-4c06-ab52-3933c2e57f5d service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:56:28.349671 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fe6563-7235-4c06-ab52-3933c2e57f5d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:56:28.357869 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fe6563-7235-4c06-ab52-3933c2e57f5d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:56:28.359282 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fe6563-7235-4c06-ab52-3933c2e57f5d service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:56:28.367976 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1297/2590] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:56:28 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 19:56:29.730618 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7abca29-fb6d-423c-b896-80adc01239e2 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:56:29.731279 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7abca29-fb6d-423c-b896-80adc01239e2 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:56:29.731279 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7abca29-fb6d-423c-b896-80adc01239e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:56:29.750205 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-f7abca29-fb6d-423c-b896-80adc01239e2 None None] MFA Rules not processed for user `8c727c852d63447d89179e17d4731628`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:56:29.751414 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-f7abca29-fb6d-423c-b896-80adc01239e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:56:29.758762 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1294/2591] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:56:29 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:56:30.005933 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fe6563-7235-4c06-ab52-3933c2e57f5d service cinder] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:56:30.007347 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-08fe6563-7235-4c06-ab52-3933c2e57f5d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:56:30.020080 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fe6563-7235-4c06-ab52-3933c2e57f5d service cinder] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:56:30.023610 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d19532e-84de-4175-9adc-f9e5274c3661 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:56:30.024228 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d19532e-84de-4175-9adc-f9e5274c3661 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:56:30.024505 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d19532e-84de-4175-9adc-f9e5274c3661 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:56:30.024774 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d19532e-84de-4175-9adc-f9e5274c3661 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:56:30.026457 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-6d19532e-84de-4175-9adc-f9e5274c3661 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:56:30.035564 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d19532e-84de-4175-9adc-f9e5274c3661 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:56:30.037015 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d19532e-84de-4175-9adc-f9e5274c3661 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:56:30.046259 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1298/2592] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:56:30 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:56:32.394106 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7abca29-fb6d-423c-b896-80adc01239e2 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:56:32.395719 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-f7abca29-fb6d-423c-b896-80adc01239e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:56:32.470598 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7abca29-fb6d-423c-b896-80adc01239e2 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:56:32.476862 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35b80ec-80e8-4fce-8752-c9e03edd578b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eef0f73297e04d62bc22acb790fad6ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:56:32.477529 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35b80ec-80e8-4fce-8752-c9e03edd578b service placement] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:56:32.477793 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35b80ec-80e8-4fce-8752-c9e03edd578b service placement] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:56:32.478067 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35b80ec-80e8-4fce-8752-c9e03edd578b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:56:32.479998 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-a35b80ec-80e8-4fce-8752-c9e03edd578b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:56:32.492740 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35b80ec-80e8-4fce-8752-c9e03edd578b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:56:32.494331 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35b80ec-80e8-4fce-8752-c9e03edd578b service placement] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:56:32.505408 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1295/2593] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:56:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:56:40.041508 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=26 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:56:42.500724 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=92 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:56:45.022226 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d19532e-84de-4175-9adc-f9e5274c3661 service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:56:45.024495 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-6d19532e-84de-4175-9adc-f9e5274c3661 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:56:45.103897 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d19532e-84de-4175-9adc-f9e5274c3661 service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:56:45.115724 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd75773-d52f-4a6e-8816-a2c0f7f269ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f43e2756b746a6825ca1a06b7d7c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:56:45.116465 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd75773-d52f-4a6e-8816-a2c0f7f269ea service nova] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:56:45.116816 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd75773-d52f-4a6e-8816-a2c0f7f269ea service nova] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:56:45.117159 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd75773-d52f-4a6e-8816-a2c0f7f269ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:56:45.119198 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-ddd75773-d52f-4a6e-8816-a2c0f7f269ea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:56:45.132012 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd75773-d52f-4a6e-8816-a2c0f7f269ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:56:45.133579 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd75773-d52f-4a6e-8816-a2c0f7f269ea service nova] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:56:45.144653 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1299/2594] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:56:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:56:52.082941 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2057eb60-8d18-461e-9ca5-936e1aafd7be None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:56:52.083407 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2057eb60-8d18-461e-9ca5-936e1aafd7be None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:56:52.083492 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2057eb60-8d18-461e-9ca5-936e1aafd7be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:56:52.102389 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-2057eb60-8d18-461e-9ca5-936e1aafd7be None None] MFA Rules not processed for user `8c727c852d63447d89179e17d4731628`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:56:52.103571 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-2057eb60-8d18-461e-9ca5-936e1aafd7be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:56:52.110849 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1296/2595] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:56:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:56:55.140204 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=34 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:57:02.097247 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:57:13.972354 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0423bd4-11cc-4add-ad0d-11ee17850dc3 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:57:13.972903 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0423bd4-11cc-4add-ad0d-11ee17850dc3 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:57:13.973244 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0423bd4-11cc-4add-ad0d-11ee17850dc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:57:14.005936 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-d0423bd4-11cc-4add-ad0d-11ee17850dc3 None None] MFA Rules not processed for user `399396a8fe254da199bc48b0ca5243d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:57:14.007160 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-d0423bd4-11cc-4add-ad0d-11ee17850dc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:57:14.067175 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1300/2596] 10.222.0.61 () {58 vars in 1009 bytes} [Mon Aug 22 19:57:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:57:14.117545 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2057eb60-8d18-461e-9ca5-936e1aafd7be None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:57:14.118990 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-2057eb60-8d18-461e-9ca5-936e1aafd7be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:57:14.194166 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2057eb60-8d18-461e-9ca5-936e1aafd7be None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:57:14.199255 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9386c6-6e12-4e58-bcf5-b0114471b6b7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2824359491934d0dacaa21a718becee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:57:14.199908 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9386c6-6e12-4e58-bcf5-b0114471b6b7 service swift] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:57:14.200188 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9386c6-6e12-4e58-bcf5-b0114471b6b7 service swift] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:57:14.201112 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9386c6-6e12-4e58-bcf5-b0114471b6b7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:57:14.209912 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9386c6-6e12-4e58-bcf5-b0114471b6b7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:57:14.211494 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9386c6-6e12-4e58-bcf5-b0114471b6b7 service swift] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:57:14.219778 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1297/2597] 10.222.0.61 () {60 vars in 1384 bytes} [Mon Aug 22 19:57:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 22 19:57:24.061024 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=29 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:57:24.216592 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=32 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:57:29.770898 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaf0069-5975-4c59-9cbd-e6a138c113c5 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:57:29.771369 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaf0069-5975-4c59-9cbd-e6a138c113c5 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:57:29.771456 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaf0069-5975-4c59-9cbd-e6a138c113c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:57:29.790394 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-3aaf0069-5975-4c59-9cbd-e6a138c113c5 None None] MFA Rules not processed for user `399396a8fe254da199bc48b0ca5243d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:57:29.791600 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-3aaf0069-5975-4c59-9cbd-e6a138c113c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:57:29.798787 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1301/2598] 10.222.0.61 () {58 vars in 967 bytes} [Mon Aug 22 19:57:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 19:57:29.811535 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9386c6-6e12-4e58-bcf5-b0114471b6b7 service swift] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:57:29.822572 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9386c6-6e12-4e58-bcf5-b0114471b6b7 service swift] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:57:29.828219 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a41285-2792-4993-b6fb-dbe7938cc795 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2824359491934d0dacaa21a718becee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:57:29.828851 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a41285-2792-4993-b6fb-dbe7938cc795 service swift] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:57:29.829129 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a41285-2792-4993-b6fb-dbe7938cc795 service swift] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:57:29.829404 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a41285-2792-4993-b6fb-dbe7938cc795 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:57:29.839941 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a41285-2792-4993-b6fb-dbe7938cc795 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:57:29.841559 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a41285-2792-4993-b6fb-dbe7938cc795 service swift] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:57:29.850511 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1298/2599] 10.222.0.61 () {60 vars in 1384 bytes} [Mon Aug 22 19:57:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 22 19:57:31.745271 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaf0069-5975-4c59-9cbd-e6a138c113c5 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:57:31.746679 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-3aaf0069-5975-4c59-9cbd-e6a138c113c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:57:31.763521 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaf0069-5975-4c59-9cbd-e6a138c113c5 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:57:31.769463 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5629b677-ec51-4da6-9c0b-338e6ad1e020 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eef0f73297e04d62bc22acb790fad6ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:57:31.770084 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5629b677-ec51-4da6-9c0b-338e6ad1e020 service placement] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:57:31.770365 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5629b677-ec51-4da6-9c0b-338e6ad1e020 service placement] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:57:31.770636 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5629b677-ec51-4da6-9c0b-338e6ad1e020 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:57:31.772453 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-5629b677-ec51-4da6-9c0b-338e6ad1e020 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:57:31.784348 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5629b677-ec51-4da6-9c0b-338e6ad1e020 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:57:31.785925 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5629b677-ec51-4da6-9c0b-338e6ad1e020 service placement] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:57:31.797015 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1302/2600] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:57:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:57:31.811265 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a41285-2792-4993-b6fb-dbe7938cc795 service swift] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:57:31.826168 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a41285-2792-4993-b6fb-dbe7938cc795 service swift] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:57:31.833062 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca02331f-bc87-4bcd-90da-ad397fe720da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:57:31.834542 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca02331f-bc87-4bcd-90da-ad397fe720da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:57:31.835069 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca02331f-bc87-4bcd-90da-ad397fe720da admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:57:31.835351 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca02331f-bc87-4bcd-90da-ad397fe720da admin admin] PATH_INFO: `/v3/users/8c727c852d63447d89179e17d4731628` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:57:31.836950 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca02331f-bc87-4bcd-90da-ad397fe720da admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c727c852d63447d89179e17d4731628)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:57:31.839361 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca02331f-bc87-4bcd-90da-ad397fe720da admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:57:31.901732 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-ca02331f-bc87-4bcd-90da-ad397fe720da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c727c852d63447d89179e17d4731628'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:57:31.902435 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-ca02331f-bc87-4bcd-90da-ad397fe720da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c727c852d63447d89179e17d4731628'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:57:31.910046 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-ca02331f-bc87-4bcd-90da-ad397fe720da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c727c852d63447d89179e17d4731628'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:57:31.918267 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-ca02331f-bc87-4bcd-90da-ad397fe720da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c727c852d63447d89179e17d4731628'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:57:31.931031 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-ca02331f-bc87-4bcd-90da-ad397fe720da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c727c852d63447d89179e17d4731628'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:57:31.937689 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1299/2601] 10.222.0.61 () {58 vars in 1224 bytes} [Mon Aug 22 19:57:31 2022] DELETE /identity/v3/users/8c727c852d63447d89179e17d4731628 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:57:32.114703 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5629b677-ec51-4da6-9c0b-338e6ad1e020 service placement] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:57:32.131230 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5629b677-ec51-4da6-9c0b-338e6ad1e020 service placement] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:57:32.193231 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bcb634-49e7-43ba-8f50-2c12b3e30cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:57:32.193787 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bcb634-49e7-43ba-8f50-2c12b3e30cca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:57:32.194060 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bcb634-49e7-43ba-8f50-2c12b3e30cca admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:57:32.194331 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bcb634-49e7-43ba-8f50-2c12b3e30cca admin admin] PATH_INFO: `/v3/projects/532a20c016764748a97e11413dab4aba` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:57:32.195441 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bcb634-49e7-43ba-8f50-2c12b3e30cca admin admin] RBAC: Authorizing `identity:delete_project(project_id=532a20c016764748a97e11413dab4aba)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:57:32.197379 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bcb634-49e7-43ba-8f50-2c12b3e30cca admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:57:32.243904 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-c1bcb634-49e7-43ba-8f50-2c12b3e30cca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '532a20c016764748a97e11413dab4aba', 'request_id': 'req-c1bcb634-49e7-43ba-8f50-2c12b3e30cca'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:57:32.251631 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-c1bcb634-49e7-43ba-8f50-2c12b3e30cca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '532a20c016764748a97e11413dab4aba', 'request_id': 'req-c1bcb634-49e7-43ba-8f50-2c12b3e30cca'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:57:32.257958 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-c1bcb634-49e7-43ba-8f50-2c12b3e30cca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '532a20c016764748a97e11413dab4aba', 'request_id': 'req-c1bcb634-49e7-43ba-8f50-2c12b3e30cca'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:57:32.270326 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-c1bcb634-49e7-43ba-8f50-2c12b3e30cca admin admin] The token cache is being invalidate because project 532a20c016764748a97e11413dab4aba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 19:57:32.270615 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-c1bcb634-49e7-43ba-8f50-2c12b3e30cca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:57:32.271300 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-c1bcb634-49e7-43ba-8f50-2c12b3e30cca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:57:32.272828 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1303/2602] 10.222.0.61 () {58 vars in 1233 bytes} [Mon Aug 22 19:57:32 2022] DELETE /identity/v3/projects/532a20c016764748a97e11413dab4aba => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:57:32.280009 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ba8943-4114-4392-b321-cb2ad8a1e128 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:57:32.280342 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ba8943-4114-4392-b321-cb2ad8a1e128 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:57:32.280656 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ba8943-4114-4392-b321-cb2ad8a1e128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:57:32.301139 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-02ba8943-4114-4392-b321-cb2ad8a1e128 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:57:32.302337 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-02ba8943-4114-4392-b321-cb2ad8a1e128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:57:32.366152 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1300/2603] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:57:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:57:32.374926 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6fe09f-b0e8-4e1f-af7c-cb27af2b27a1 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:57:32.375223 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6fe09f-b0e8-4e1f-af7c-cb27af2b27a1 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:57:32.375622 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6fe09f-b0e8-4e1f-af7c-cb27af2b27a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:57:32.397478 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-6b6fe09f-b0e8-4e1f-af7c-cb27af2b27a1 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:57:32.398710 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-6b6fe09f-b0e8-4e1f-af7c-cb27af2b27a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:57:32.407458 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1304/2604] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:57:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:57:32.411884 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ba8943-4114-4392-b321-cb2ad8a1e128 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:57:32.421918 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ba8943-4114-4392-b321-cb2ad8a1e128 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:57:32.426603 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7889cce-c8af-47d1-94b0-be5223f8a831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:57:32.427204 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7889cce-c8af-47d1-94b0-be5223f8a831 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:57:32.427483 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7889cce-c8af-47d1-94b0-be5223f8a831 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:57:32.427788 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7889cce-c8af-47d1-94b0-be5223f8a831 admin admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:57:32.428437 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7889cce-c8af-47d1-94b0-be5223f8a831 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 19:57:32.428739 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7889cce-c8af-47d1-94b0-be5223f8a831 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:57:32.430328 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7889cce-c8af-47d1-94b0-be5223f8a831 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:57:32.442194 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1301/2605] 10.222.0.61 () {58 vars in 1153 bytes} [Mon Aug 22 19:57:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 19:57:32.446672 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6fe09f-b0e8-4e1f-af7c-cb27af2b27a1 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:57:32.456793 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6fe09f-b0e8-4e1f-af7c-cb27af2b27a1 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:57:32.461854 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137a2884-8d12-4c25-9003-799b4bda00c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:57:32.462458 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137a2884-8d12-4c25-9003-799b4bda00c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:57:32.462738 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137a2884-8d12-4c25-9003-799b4bda00c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:57:32.463012 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137a2884-8d12-4c25-9003-799b4bda00c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:57:32.463695 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137a2884-8d12-4c25-9003-799b4bda00c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:57:32.465627 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137a2884-8d12-4c25-9003-799b4bda00c0 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:57:32.494026 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1305/2606] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 19:57:32 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:57:32.498222 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7889cce-c8af-47d1-94b0-be5223f8a831 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:57:32.508237 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7889cce-c8af-47d1-94b0-be5223f8a831 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:57:32.555734 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:57:32.570977 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec902b9c-42b9-4c50-a4d3-1c5f157e89cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:57:32.571610 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec902b9c-42b9-4c50-a4d3-1c5f157e89cb admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:57:32.571887 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec902b9c-42b9-4c50-a4d3-1c5f157e89cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:57:32.572236 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec902b9c-42b9-4c50-a4d3-1c5f157e89cb admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:57:32.572886 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec902b9c-42b9-4c50-a4d3-1c5f157e89cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:57:32.574822 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec902b9c-42b9-4c50-a4d3-1c5f157e89cb admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:57:32.599215 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1302/2607] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 19:57:32 2022] POST /identity/v3/users => generated 286 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 19:57:32.603210 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137a2884-8d12-4c25-9003-799b4bda00c0 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:57:32.614059 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137a2884-8d12-4c25-9003-799b4bda00c0 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:57:32.618800 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704536b9-123e-4321-a6a1-85a355c7d061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:57:32.619413 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704536b9-123e-4321-a6a1-85a355c7d061 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:57:32.619718 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704536b9-123e-4321-a6a1-85a355c7d061 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:57:32.619992 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704536b9-123e-4321-a6a1-85a355c7d061 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:57:32.620638 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704536b9-123e-4321-a6a1-85a355c7d061 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:57:32.622248 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704536b9-123e-4321-a6a1-85a355c7d061 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:57:32.632718 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1306/2608] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:57:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:57:32.636748 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec902b9c-42b9-4c50-a4d3-1c5f157e89cb admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:57:32.647627 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec902b9c-42b9-4c50-a4d3-1c5f157e89cb admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:57:32.654117 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d325a28e-0daa-434c-a531-bdbd17d30ca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:57:32.654735 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d325a28e-0daa-434c-a531-bdbd17d30ca4 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:57:32.655048 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d325a28e-0daa-434c-a531-bdbd17d30ca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:57:32.655299 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d325a28e-0daa-434c-a531-bdbd17d30ca4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:57:32.655986 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d325a28e-0daa-434c-a531-bdbd17d30ca4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:57:32.657576 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d325a28e-0daa-434c-a531-bdbd17d30ca4 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:57:32.669193 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1303/2609] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 19:57:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 19:57:32.673590 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704536b9-123e-4321-a6a1-85a355c7d061 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:57:32.684213 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704536b9-123e-4321-a6a1-85a355c7d061 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:57:32.689262 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f1de09-581d-4539-b407-f4ae2a9fc7de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:57:32.690016 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f1de09-581d-4539-b407-f4ae2a9fc7de admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:57:32.690367 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f1de09-581d-4539-b407-f4ae2a9fc7de admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:57:32.690963 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f1de09-581d-4539-b407-f4ae2a9fc7de admin admin] PATH_INFO: `/v3/projects/32a3494067924a16a6d2f83d3e6ae432/users/0f892f662e41437e812c055038118c40/roles/74a40bc819cb42eea7e966a4b2a016f1` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:57:32.712142 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f1de09-581d-4539-b407-f4ae2a9fc7de admin admin] RBAC: Authorizing `identity:create_grant(project_id=32a3494067924a16a6d2f83d3e6ae432, user_id=0f892f662e41437e812c055038118c40, role_id=74a40bc819cb42eea7e966a4b2a016f1)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:57:32.714745 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f1de09-581d-4539-b407-f4ae2a9fc7de admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:57:32.729789 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1307/2610] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 19:57:32 2022] PUT /identity/v3/projects/32a3494067924a16a6d2f83d3e6ae432/users/0f892f662e41437e812c055038118c40/roles/74a40bc819cb42eea7e966a4b2a016f1 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:57:32.741952 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6280c7eb-97e4-47d5-9646-21f75159d37d None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:57:32.742249 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6280c7eb-97e4-47d5-9646-21f75159d37d None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:57:32.742531 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6280c7eb-97e4-47d5-9646-21f75159d37d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:57:32.778033 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-6280c7eb-97e4-47d5-9646-21f75159d37d None None] MFA Rules not processed for user `0f892f662e41437e812c055038118c40`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:57:32.779303 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-6280c7eb-97e4-47d5-9646-21f75159d37d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:57:32.847400 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1304/2611] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:57:32 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:57:32.858795 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f1de09-581d-4539-b407-f4ae2a9fc7de admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:57:32.860544 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-17f1de09-581d-4539-b407-f4ae2a9fc7de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:57:32.870556 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:57:32.927098 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f1de09-581d-4539-b407-f4ae2a9fc7de admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:57:32.931309 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7ff2ed-d1de-4962-a026-dd4a96bedfc2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e79498873d4ed29a0e19bbea9b9214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:57:32.931968 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7ff2ed-d1de-4962-a026-dd4a96bedfc2 service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:57:32.932249 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7ff2ed-d1de-4962-a026-dd4a96bedfc2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:57:32.932525 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7ff2ed-d1de-4962-a026-dd4a96bedfc2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:57:32.942613 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7ff2ed-d1de-4962-a026-dd4a96bedfc2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:57:32.944110 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7ff2ed-d1de-4962-a026-dd4a96bedfc2 service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:57:32.953769 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1308/2612] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:57:32 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:57:32.978886 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6280c7eb-97e4-47d5-9646-21f75159d37d None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:57:33.044498 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6280c7eb-97e4-47d5-9646-21f75159d37d None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:57:33.049122 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21052fc-1bfb-4753-b611-f4188562ec3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:57:33.049750 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21052fc-1bfb-4753-b611-f4188562ec3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:57:33.050031 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21052fc-1bfb-4753-b611-f4188562ec3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:57:33.050305 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21052fc-1bfb-4753-b611-f4188562ec3a admin admin] PATH_INFO: `/v3/users/0f892f662e41437e812c055038118c40` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:57:33.051938 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21052fc-1bfb-4753-b611-f4188562ec3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f892f662e41437e812c055038118c40)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:57:33.053847 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21052fc-1bfb-4753-b611-f4188562ec3a admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:57:33.105250 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a21052fc-1bfb-4753-b611-f4188562ec3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f892f662e41437e812c055038118c40'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:57:33.105924 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a21052fc-1bfb-4753-b611-f4188562ec3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f892f662e41437e812c055038118c40'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:57:33.113668 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a21052fc-1bfb-4753-b611-f4188562ec3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f892f662e41437e812c055038118c40'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:57:33.115375 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:57:33.122625 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a21052fc-1bfb-4753-b611-f4188562ec3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f892f662e41437e812c055038118c40'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:57:33.136031 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a21052fc-1bfb-4753-b611-f4188562ec3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f892f662e41437e812c055038118c40'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:57:33.142534 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1305/2613] 10.222.0.61 () {58 vars in 1224 bytes} [Mon Aug 22 19:57:32 2022] DELETE /identity/v3/users/0f892f662e41437e812c055038118c40 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:57:33.147397 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc92b54-7c64-46ac-84fb-8c58623db4d9 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:57:33.147397 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc92b54-7c64-46ac-84fb-8c58623db4d9 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:57:33.147619 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc92b54-7c64-46ac-84fb-8c58623db4d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:57:33.170890 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-fdc92b54-7c64-46ac-84fb-8c58623db4d9 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 19:57:33.172131 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-fdc92b54-7c64-46ac-84fb-8c58623db4d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:57:33.238819 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1309/2614] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 19:57:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 19:57:33.250175 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21052fc-1bfb-4753-b611-f4188562ec3a admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:57:33.251672 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-a21052fc-1bfb-4753-b611-f4188562ec3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:57:33.310179 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21052fc-1bfb-4753-b611-f4188562ec3a admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:57:33.313878 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee8b2c0-ff05-436a-8a7a-da601d1dd4c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:57:33.314487 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee8b2c0-ff05-436a-8a7a-da601d1dd4c4 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:57:33.314762 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee8b2c0-ff05-436a-8a7a-da601d1dd4c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:57:33.315030 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee8b2c0-ff05-436a-8a7a-da601d1dd4c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:57:33.324299 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee8b2c0-ff05-436a-8a7a-da601d1dd4c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:57:33.325751 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee8b2c0-ff05-436a-8a7a-da601d1dd4c4 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:57:33.335520 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1306/2615] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:57:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:57:33.615663 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc92b54-7c64-46ac-84fb-8c58623db4d9 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:57:33.617421 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-fdc92b54-7c64-46ac-84fb-8c58623db4d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:57:33.632663 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc92b54-7c64-46ac-84fb-8c58623db4d9 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:57:33.637635 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a828906d-cbc1-4865-8def-3d50a137ba72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:57:33.638296 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a828906d-cbc1-4865-8def-3d50a137ba72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:57:33.638724 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a828906d-cbc1-4865-8def-3d50a137ba72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:57:33.639233 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a828906d-cbc1-4865-8def-3d50a137ba72 admin admin] PATH_INFO: `/v3/projects/32a3494067924a16a6d2f83d3e6ae432` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:57:33.640637 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a828906d-cbc1-4865-8def-3d50a137ba72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=32a3494067924a16a6d2f83d3e6ae432)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:57:33.642621 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a828906d-cbc1-4865-8def-3d50a137ba72 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:57:33.704998 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-a828906d-cbc1-4865-8def-3d50a137ba72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32a3494067924a16a6d2f83d3e6ae432', 'request_id': 'req-a828906d-cbc1-4865-8def-3d50a137ba72'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:57:33.712749 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-a828906d-cbc1-4865-8def-3d50a137ba72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32a3494067924a16a6d2f83d3e6ae432', 'request_id': 'req-a828906d-cbc1-4865-8def-3d50a137ba72'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:57:33.720352 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-a828906d-cbc1-4865-8def-3d50a137ba72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32a3494067924a16a6d2f83d3e6ae432', 'request_id': 'req-a828906d-cbc1-4865-8def-3d50a137ba72'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:57:33.740397 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-a828906d-cbc1-4865-8def-3d50a137ba72 admin admin] The token cache is being invalidate because project 32a3494067924a16a6d2f83d3e6ae432 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 19:57:33.740725 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-a828906d-cbc1-4865-8def-3d50a137ba72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:57:33.741364 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-a828906d-cbc1-4865-8def-3d50a137ba72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 19:57:33.742903 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1310/2616] 10.222.0.61 () {58 vars in 1233 bytes} [Mon Aug 22 19:57:33 2022] DELETE /identity/v3/projects/32a3494067924a16a6d2f83d3e6ae432 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 19:57:43.331058 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=34 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:57:43.731694 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=77,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 19:57:54.442834 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee8b2c0-ff05-436a-8a7a-da601d1dd4c4 service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 19:57:54.444271 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-8ee8b2c0-ff05-436a-8a7a-da601d1dd4c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:57:54.573283 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee8b2c0-ff05-436a-8a7a-da601d1dd4c4 service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 19:57:54.639408 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7e708f-cb79-4997-8c28-c7b04ec46b52 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eef0f73297e04d62bc22acb790fad6ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 19:57:54.640161 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7e708f-cb79-4997-8c28-c7b04ec46b52 service placement] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 19:57:54.640442 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7e708f-cb79-4997-8c28-c7b04ec46b52 service placement] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 19:57:54.640746 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7e708f-cb79-4997-8c28-c7b04ec46b52 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 19:57:54.642550 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-7d7e708f-cb79-4997-8c28-c7b04ec46b52 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 19:57:54.654648 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7e708f-cb79-4997-8c28-c7b04ec46b52 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 19:57:54.656274 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7e708f-cb79-4997-8c28-c7b04ec46b52 service placement] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 19:57:54.666822 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1307/2617] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 19:57:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 19:58:04.662215 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=80 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:00:17.095237 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a828906d-cbc1-4865-8def-3d50a137ba72 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:00:17.096668 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-a828906d-cbc1-4865-8def-3d50a137ba72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:00:17.172341 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a828906d-cbc1-4865-8def-3d50a137ba72 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:00:17.175993 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a3370c-01da-40b7-9a53-75b4a066ac49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:00:17.176598 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a3370c-01da-40b7-9a53-75b4a066ac49 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:00:17.176877 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a3370c-01da-40b7-9a53-75b4a066ac49 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:00:17.177150 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a3370c-01da-40b7-9a53-75b4a066ac49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:00:17.178994 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-07a3370c-01da-40b7-9a53-75b4a066ac49 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:00:17.245433 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a3370c-01da-40b7-9a53-75b4a066ac49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:00:17.246779 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a3370c-01da-40b7-9a53-75b4a066ac49 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:00:17.257031 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1311/2618] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 20:00:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 20:00:20.144217 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7e708f-cb79-4997-8c28-c7b04ec46b52 service placement] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:00:20.144217 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-7d7e708f-cb79-4997-8c28-c7b04ec46b52 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:00:20.174599 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7e708f-cb79-4997-8c28-c7b04ec46b52 service placement] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:00:20.179535 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2e3ade-df4f-4a25-85dd-24f5cc218dfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:00:20.180215 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2e3ade-df4f-4a25-85dd-24f5cc218dfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:00:20.180548 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2e3ade-df4f-4a25-85dd-24f5cc218dfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:00:20.180828 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2e3ade-df4f-4a25-85dd-24f5cc218dfc admin admin] PATH_INFO: `/v3/users/61388384a318473e987ab06426509b69` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:00:20.181980 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2e3ade-df4f-4a25-85dd-24f5cc218dfc admin admin] RBAC: Authorizing `identity:delete_user(user_id=61388384a318473e987ab06426509b69)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:00:20.183949 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2e3ade-df4f-4a25-85dd-24f5cc218dfc admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:00:20.237766 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-be2e3ade-df4f-4a25-85dd-24f5cc218dfc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61388384a318473e987ab06426509b69'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:00:20.238419 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-be2e3ade-df4f-4a25-85dd-24f5cc218dfc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61388384a318473e987ab06426509b69'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:00:20.245690 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-be2e3ade-df4f-4a25-85dd-24f5cc218dfc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61388384a318473e987ab06426509b69'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:00:20.255373 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-be2e3ade-df4f-4a25-85dd-24f5cc218dfc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61388384a318473e987ab06426509b69'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:00:20.273278 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-be2e3ade-df4f-4a25-85dd-24f5cc218dfc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61388384a318473e987ab06426509b69'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:00:20.279615 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1308/2619] 10.222.0.61 () {58 vars in 1224 bytes} [Mon Aug 22 20:00:20 2022] DELETE /identity/v3/users/61388384a318473e987ab06426509b69 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 20:00:20.668567 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a3370c-01da-40b7-9a53-75b4a066ac49 service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:00:20.683118 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a3370c-01da-40b7-9a53-75b4a066ac49 service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:00:20.741883 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d598464-abe9-4304-99a1-b7cad8e4db62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:00:20.742521 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d598464-abe9-4304-99a1-b7cad8e4db62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:00:20.742800 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d598464-abe9-4304-99a1-b7cad8e4db62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:00:20.743072 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d598464-abe9-4304-99a1-b7cad8e4db62 admin admin] PATH_INFO: `/v3/projects/1e2b975d3d2d45d384b5a8dd92da782f` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:00:20.753330 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d598464-abe9-4304-99a1-b7cad8e4db62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e2b975d3d2d45d384b5a8dd92da782f)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:00:20.755292 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d598464-abe9-4304-99a1-b7cad8e4db62 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:00:20.781024 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:00:20.804076 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-1d598464-abe9-4304-99a1-b7cad8e4db62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e2b975d3d2d45d384b5a8dd92da782f', 'request_id': 'req-1d598464-abe9-4304-99a1-b7cad8e4db62'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:00:20.812147 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-1d598464-abe9-4304-99a1-b7cad8e4db62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e2b975d3d2d45d384b5a8dd92da782f', 'request_id': 'req-1d598464-abe9-4304-99a1-b7cad8e4db62'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:00:20.818940 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-1d598464-abe9-4304-99a1-b7cad8e4db62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e2b975d3d2d45d384b5a8dd92da782f', 'request_id': 'req-1d598464-abe9-4304-99a1-b7cad8e4db62'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:00:20.829271 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-1d598464-abe9-4304-99a1-b7cad8e4db62 admin admin] The token cache is being invalidate because project 1e2b975d3d2d45d384b5a8dd92da782f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 20:00:20.829557 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-1d598464-abe9-4304-99a1-b7cad8e4db62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:00:20.830447 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-1d598464-abe9-4304-99a1-b7cad8e4db62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:00:20.831976 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1312/2620] 10.222.0.61 () {58 vars in 1233 bytes} [Mon Aug 22 20:00:20 2022] DELETE /identity/v3/projects/1e2b975d3d2d45d384b5a8dd92da782f => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 20:00:20.837782 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495be449-5a0b-4255-9762-3fc3d35512c2 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:00:20.838066 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495be449-5a0b-4255-9762-3fc3d35512c2 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:00:20.838339 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495be449-5a0b-4255-9762-3fc3d35512c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:00:20.858808 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-495be449-5a0b-4255-9762-3fc3d35512c2 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 20:00:20.860181 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-495be449-5a0b-4255-9762-3fc3d35512c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:00:20.922714 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1309/2621] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 20:00:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 20:00:20.930103 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2cba3d-9fee-48f6-9227-90003b9e78df None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:00:20.930390 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2cba3d-9fee-48f6-9227-90003b9e78df None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:00:20.930665 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2cba3d-9fee-48f6-9227-90003b9e78df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:00:20.949752 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-4a2cba3d-9fee-48f6-9227-90003b9e78df None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 20:00:20.951482 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-4a2cba3d-9fee-48f6-9227-90003b9e78df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:00:20.959704 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1313/2622] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 20:00:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 20:00:20.963937 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495be449-5a0b-4255-9762-3fc3d35512c2 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:00:20.974340 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495be449-5a0b-4255-9762-3fc3d35512c2 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:00:20.978871 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437165e4-dffc-4990-b8bb-98aa902e32af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:00:20.979458 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437165e4-dffc-4990-b8bb-98aa902e32af admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:00:20.979767 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437165e4-dffc-4990-b8bb-98aa902e32af admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:00:20.980037 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437165e4-dffc-4990-b8bb-98aa902e32af admin admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:00:20.980684 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437165e4-dffc-4990-b8bb-98aa902e32af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 20:00:20.980983 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437165e4-dffc-4990-b8bb-98aa902e32af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:00:20.982640 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437165e4-dffc-4990-b8bb-98aa902e32af admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:00:20.994186 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1310/2623] 10.222.0.61 () {58 vars in 1153 bytes} [Mon Aug 22 20:00:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 20:00:20.998209 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2cba3d-9fee-48f6-9227-90003b9e78df None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:00:21.007485 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2cba3d-9fee-48f6-9227-90003b9e78df None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:00:21.012300 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca68415c-1390-49f7-a4f5-4e3c0cbcc76a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:00:21.012936 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca68415c-1390-49f7-a4f5-4e3c0cbcc76a admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:00:21.013760 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca68415c-1390-49f7-a4f5-4e3c0cbcc76a admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:00:21.014036 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca68415c-1390-49f7-a4f5-4e3c0cbcc76a admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:00:21.014680 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca68415c-1390-49f7-a4f5-4e3c0cbcc76a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:00:21.016664 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca68415c-1390-49f7-a4f5-4e3c0cbcc76a admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:00:21.046936 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1314/2624] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 20:00:20 2022] POST /identity/v3/projects => generated 355 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 20:00:21.051220 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437165e4-dffc-4990-b8bb-98aa902e32af admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:00:21.061723 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437165e4-dffc-4990-b8bb-98aa902e32af admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:00:21.108493 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:00:21.123006 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a787288-e84f-4ab8-9d5e-6def845353db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:00:21.123636 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a787288-e84f-4ab8-9d5e-6def845353db admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:00:21.123927 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a787288-e84f-4ab8-9d5e-6def845353db admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:00:21.124201 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a787288-e84f-4ab8-9d5e-6def845353db admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:00:21.124844 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a787288-e84f-4ab8-9d5e-6def845353db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:00:21.126736 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a787288-e84f-4ab8-9d5e-6def845353db admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:00:21.149942 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1311/2625] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 20:00:21 2022] POST /identity/v3/users => generated 278 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 20:00:21.154584 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca68415c-1390-49f7-a4f5-4e3c0cbcc76a admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:00:21.164933 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca68415c-1390-49f7-a4f5-4e3c0cbcc76a admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:00:21.169573 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cea00b5-dcdc-4e3f-91ac-d26ce58f605e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:00:21.170239 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cea00b5-dcdc-4e3f-91ac-d26ce58f605e admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:00:21.170443 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cea00b5-dcdc-4e3f-91ac-d26ce58f605e admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:00:21.170710 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cea00b5-dcdc-4e3f-91ac-d26ce58f605e admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:00:21.171345 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cea00b5-dcdc-4e3f-91ac-d26ce58f605e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:00:21.173057 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cea00b5-dcdc-4e3f-91ac-d26ce58f605e admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:00:21.182893 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1315/2626] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 20:00:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 20:00:21.186775 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a787288-e84f-4ab8-9d5e-6def845353db admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:00:21.196799 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a787288-e84f-4ab8-9d5e-6def845353db admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:00:21.201996 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800048e6-cbfc-4c5b-b703-6ad92a6a754f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:00:21.202630 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800048e6-cbfc-4c5b-b703-6ad92a6a754f admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:00:21.202911 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800048e6-cbfc-4c5b-b703-6ad92a6a754f admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:00:21.203196 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800048e6-cbfc-4c5b-b703-6ad92a6a754f admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:00:21.204090 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800048e6-cbfc-4c5b-b703-6ad92a6a754f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:00:21.206068 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800048e6-cbfc-4c5b-b703-6ad92a6a754f admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:00:21.217304 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1312/2627] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 20:00:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 20:00:21.221434 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cea00b5-dcdc-4e3f-91ac-d26ce58f605e admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:00:21.231666 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cea00b5-dcdc-4e3f-91ac-d26ce58f605e admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:00:21.236377 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1025d6c2-0d77-432c-8539-bd1b8337ab17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:00:21.237073 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1025d6c2-0d77-432c-8539-bd1b8337ab17 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:00:21.237354 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1025d6c2-0d77-432c-8539-bd1b8337ab17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:00:21.237627 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1025d6c2-0d77-432c-8539-bd1b8337ab17 admin admin] PATH_INFO: `/v3/projects/36d0110a9d724746835cb719d0394242/users/64242d329d9f4560a871314582707a01/roles/74a40bc819cb42eea7e966a4b2a016f1` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:00:21.252403 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1025d6c2-0d77-432c-8539-bd1b8337ab17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36d0110a9d724746835cb719d0394242, user_id=64242d329d9f4560a871314582707a01, role_id=74a40bc819cb42eea7e966a4b2a016f1)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:00:21.255074 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1025d6c2-0d77-432c-8539-bd1b8337ab17 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:00:21.269189 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1316/2628] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 20:00:21 2022] PUT /identity/v3/projects/36d0110a9d724746835cb719d0394242/users/64242d329d9f4560a871314582707a01/roles/74a40bc819cb42eea7e966a4b2a016f1 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 20:00:21.279552 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610f4359-16fe-4b78-8ccd-42d6bb53b800 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:00:21.279927 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610f4359-16fe-4b78-8ccd-42d6bb53b800 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:00:21.280206 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610f4359-16fe-4b78-8ccd-42d6bb53b800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:00:21.310107 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-610f4359-16fe-4b78-8ccd-42d6bb53b800 None None] MFA Rules not processed for user `64242d329d9f4560a871314582707a01`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 20:00:21.311346 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-610f4359-16fe-4b78-8ccd-42d6bb53b800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:00:21.376271 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1313/2629] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 20:00:21 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 20:00:21.380743 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1025d6c2-0d77-432c-8539-bd1b8337ab17 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:00:21.441425 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1025d6c2-0d77-432c-8539-bd1b8337ab17 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:00:21.446041 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175f14fc-6ed4-4f11-b43b-31f95f1fab51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:00:21.446640 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175f14fc-6ed4-4f11-b43b-31f95f1fab51 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:00:21.446931 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175f14fc-6ed4-4f11-b43b-31f95f1fab51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:00:21.447181 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175f14fc-6ed4-4f11-b43b-31f95f1fab51 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:00:21.448073 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175f14fc-6ed4-4f11-b43b-31f95f1fab51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:00:21.450063 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175f14fc-6ed4-4f11-b43b-31f95f1fab51 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:00:21.476694 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1317/2630] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 20:00:21 2022] POST /identity/v3/projects => generated 355 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 20:00:21.480735 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610f4359-16fe-4b78-8ccd-42d6bb53b800 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:00:21.491432 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610f4359-16fe-4b78-8ccd-42d6bb53b800 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:00:21.546583 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af414df0-61b2-47ad-9efb-e19b5014fd15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:00:21.547184 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af414df0-61b2-47ad-9efb-e19b5014fd15 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:00:21.547459 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af414df0-61b2-47ad-9efb-e19b5014fd15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:00:21.547758 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af414df0-61b2-47ad-9efb-e19b5014fd15 admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:00:21.548379 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af414df0-61b2-47ad-9efb-e19b5014fd15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:00:21.550268 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af414df0-61b2-47ad-9efb-e19b5014fd15 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:00:21.571107 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1314/2631] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 20:00:21 2022] POST /identity/v3/users => generated 284 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 20:00:21.575211 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175f14fc-6ed4-4f11-b43b-31f95f1fab51 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:00:21.587135 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175f14fc-6ed4-4f11-b43b-31f95f1fab51 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:00:21.591696 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e82047-7db6-466a-8183-856eabec662f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:00:21.592289 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e82047-7db6-466a-8183-856eabec662f admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:00:21.592562 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e82047-7db6-466a-8183-856eabec662f admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:00:21.592827 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e82047-7db6-466a-8183-856eabec662f admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:00:21.593456 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e82047-7db6-466a-8183-856eabec662f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:00:21.595043 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e82047-7db6-466a-8183-856eabec662f admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:00:21.605911 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1318/2632] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 20:00:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 20:00:21.609965 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af414df0-61b2-47ad-9efb-e19b5014fd15 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:00:21.620037 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af414df0-61b2-47ad-9efb-e19b5014fd15 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:00:21.625222 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a150cda-0584-4a6d-82f2-e53683419499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:00:21.625876 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a150cda-0584-4a6d-82f2-e53683419499 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:00:21.626155 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a150cda-0584-4a6d-82f2-e53683419499 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:00:21.626431 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a150cda-0584-4a6d-82f2-e53683419499 admin admin] PATH_INFO: `/v3/projects/a7136bdc811d44179225e35e80932926/users/bd56880cb40c4727849680fbb2689188/roles/163c30c3b3c246beb849ce176f02d9da` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:00:21.641533 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a150cda-0584-4a6d-82f2-e53683419499 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7136bdc811d44179225e35e80932926, user_id=bd56880cb40c4727849680fbb2689188, role_id=163c30c3b3c246beb849ce176f02d9da)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:00:21.644210 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a150cda-0584-4a6d-82f2-e53683419499 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:00:21.657106 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1315/2633] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 20:00:21 2022] PUT /identity/v3/projects/a7136bdc811d44179225e35e80932926/users/bd56880cb40c4727849680fbb2689188/roles/163c30c3b3c246beb849ce176f02d9da => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 20:00:21.661211 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e82047-7db6-466a-8183-856eabec662f admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:00:21.671165 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e82047-7db6-466a-8183-856eabec662f admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:00:21.689413 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:00:21.729793 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9e07f2-bb55-484b-8864-f076761f1fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:00:21.730319 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9e07f2-bb55-484b-8864-f076761f1fdd admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:00:21.730593 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9e07f2-bb55-484b-8864-f076761f1fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:00:21.730860 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9e07f2-bb55-484b-8864-f076761f1fdd admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:00:21.731498 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9e07f2-bb55-484b-8864-f076761f1fdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:00:21.733670 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9e07f2-bb55-484b-8864-f076761f1fdd admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:00:21.743253 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1319/2634] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 20:00:21 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 20:00:21.747557 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a150cda-0584-4a6d-82f2-e53683419499 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:00:21.758469 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a150cda-0584-4a6d-82f2-e53683419499 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:00:21.762987 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e252117-2706-47c0-a3ce-255edf6226e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:00:21.763698 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e252117-2706-47c0-a3ce-255edf6226e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:00:21.763974 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e252117-2706-47c0-a3ce-255edf6226e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:00:21.764243 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e252117-2706-47c0-a3ce-255edf6226e4 admin admin] PATH_INFO: `/v3/projects/a7136bdc811d44179225e35e80932926/users/bd56880cb40c4727849680fbb2689188/roles/163c30c3b3c246beb849ce176f02d9da` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:00:21.766132 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e252117-2706-47c0-a3ce-255edf6226e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7136bdc811d44179225e35e80932926, user_id=bd56880cb40c4727849680fbb2689188, role_id=163c30c3b3c246beb849ce176f02d9da)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:00:21.768763 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e252117-2706-47c0-a3ce-255edf6226e4 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:00:21.779147 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1316/2635] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 20:00:21 2022] PUT /identity/v3/projects/a7136bdc811d44179225e35e80932926/users/bd56880cb40c4727849680fbb2689188/roles/163c30c3b3c246beb849ce176f02d9da => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 20:00:21.790159 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea04bd0-39b8-4f16-b961-9ab99bd8f4fa None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:00:21.790449 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea04bd0-39b8-4f16-b961-9ab99bd8f4fa None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:00:21.790732 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea04bd0-39b8-4f16-b961-9ab99bd8f4fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:00:21.819503 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-eea04bd0-39b8-4f16-b961-9ab99bd8f4fa None None] MFA Rules not processed for user `bd56880cb40c4727849680fbb2689188`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 20:00:21.820768 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-eea04bd0-39b8-4f16-b961-9ab99bd8f4fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:00:21.889159 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1320/2636] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 20:00:21 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 20:00:21.902819 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e252117-2706-47c0-a3ce-255edf6226e4 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:00:21.905168 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-8e252117-2706-47c0-a3ce-255edf6226e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:00:21.917901 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:00:21.980084 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e252117-2706-47c0-a3ce-255edf6226e4 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:00:21.984745 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234cdb3a-7ca4-4c88-9ea7-6f68b50abaea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e79498873d4ed29a0e19bbea9b9214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:00:21.985364 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234cdb3a-7ca4-4c88-9ea7-6f68b50abaea service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:00:21.985641 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234cdb3a-7ca4-4c88-9ea7-6f68b50abaea service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:00:21.985913 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234cdb3a-7ca4-4c88-9ea7-6f68b50abaea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:00:21.995905 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234cdb3a-7ca4-4c88-9ea7-6f68b50abaea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:00:21.997345 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234cdb3a-7ca4-4c88-9ea7-6f68b50abaea service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:00:22.007757 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1317/2637] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 20:00:21 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 20:00:22.088189 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea04bd0-39b8-4f16-b961-9ab99bd8f4fa None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:00:22.099222 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea04bd0-39b8-4f16-b961-9ab99bd8f4fa None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:00:22.103084 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499ed48d-f73c-483a-b641-7f15b06fc9db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e79498873d4ed29a0e19bbea9b9214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:00:22.103556 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499ed48d-f73c-483a-b641-7f15b06fc9db service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:00:22.103854 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499ed48d-f73c-483a-b641-7f15b06fc9db service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:00:22.104308 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499ed48d-f73c-483a-b641-7f15b06fc9db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:00:22.168979 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499ed48d-f73c-483a-b641-7f15b06fc9db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:00:22.171009 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499ed48d-f73c-483a-b641-7f15b06fc9db service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:00:22.180913 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1321/2638] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 20:00:22 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 20:00:32.004365 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=27 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:00:32.176415 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=81 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:00:37.182275 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234cdb3a-7ca4-4c88-9ea7-6f68b50abaea service cinder] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:00:37.250522 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234cdb3a-7ca4-4c88-9ea7-6f68b50abaea service cinder] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:00:37.255115 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41b8813-966b-449d-9f09-4e9284cfd507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:00:37.255826 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41b8813-966b-449d-9f09-4e9284cfd507 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:00:37.256133 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41b8813-966b-449d-9f09-4e9284cfd507 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:00:37.256407 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41b8813-966b-449d-9f09-4e9284cfd507 admin admin] PATH_INFO: `/v3/users/64242d329d9f4560a871314582707a01` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:00:37.257550 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41b8813-966b-449d-9f09-4e9284cfd507 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64242d329d9f4560a871314582707a01)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:00:37.259474 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41b8813-966b-449d-9f09-4e9284cfd507 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:00:37.311175 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-d41b8813-966b-449d-9f09-4e9284cfd507 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64242d329d9f4560a871314582707a01'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:00:37.311866 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-d41b8813-966b-449d-9f09-4e9284cfd507 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64242d329d9f4560a871314582707a01'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:00:37.319032 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-d41b8813-966b-449d-9f09-4e9284cfd507 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64242d329d9f4560a871314582707a01'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:00:37.328139 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-d41b8813-966b-449d-9f09-4e9284cfd507 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64242d329d9f4560a871314582707a01'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:00:37.341166 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-d41b8813-966b-449d-9f09-4e9284cfd507 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64242d329d9f4560a871314582707a01'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:00:37.347564 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1318/2639] 10.222.0.61 () {58 vars in 1224 bytes} [Mon Aug 22 20:00:37 2022] DELETE /identity/v3/users/64242d329d9f4560a871314582707a01 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 20:00:37.352025 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac74f7d1-e058-4ad3-b52b-0b368214f2fe None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:00:37.352527 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac74f7d1-e058-4ad3-b52b-0b368214f2fe None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:00:37.352626 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac74f7d1-e058-4ad3-b52b-0b368214f2fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:00:37.374505 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-ac74f7d1-e058-4ad3-b52b-0b368214f2fe None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 20:00:37.376080 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-ac74f7d1-e058-4ad3-b52b-0b368214f2fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:00:37.445562 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1322/2640] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 20:00:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 20:00:37.456250 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41b8813-966b-449d-9f09-4e9284cfd507 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:00:37.457685 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-d41b8813-966b-449d-9f09-4e9284cfd507 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:00:37.528901 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41b8813-966b-449d-9f09-4e9284cfd507 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:00:37.533448 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba50121-8ddb-4267-809d-90615f51f00f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:00:37.534066 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba50121-8ddb-4267-809d-90615f51f00f service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:00:37.534344 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba50121-8ddb-4267-809d-90615f51f00f service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:00:37.534617 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba50121-8ddb-4267-809d-90615f51f00f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:00:37.544683 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba50121-8ddb-4267-809d-90615f51f00f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:00:37.546146 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba50121-8ddb-4267-809d-90615f51f00f service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:00:37.556063 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1319/2641] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 20:00:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 20:00:37.856272 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac74f7d1-e058-4ad3-b52b-0b368214f2fe None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:00:37.856272 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-ac74f7d1-e058-4ad3-b52b-0b368214f2fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:00:37.866667 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac74f7d1-e058-4ad3-b52b-0b368214f2fe None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:00:37.871576 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b069ee-581e-474d-b24d-8d8e888e3d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:00:37.872244 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b069ee-581e-474d-b24d-8d8e888e3d59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:00:37.872520 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b069ee-581e-474d-b24d-8d8e888e3d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:00:37.872789 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b069ee-581e-474d-b24d-8d8e888e3d59 admin admin] PATH_INFO: `/v3/projects/36d0110a9d724746835cb719d0394242` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:00:37.874094 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b069ee-581e-474d-b24d-8d8e888e3d59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=36d0110a9d724746835cb719d0394242)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:00:37.877480 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b069ee-581e-474d-b24d-8d8e888e3d59 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:00:37.937616 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-79b069ee-581e-474d-b24d-8d8e888e3d59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36d0110a9d724746835cb719d0394242', 'request_id': 'req-79b069ee-581e-474d-b24d-8d8e888e3d59'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:00:37.946121 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-79b069ee-581e-474d-b24d-8d8e888e3d59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36d0110a9d724746835cb719d0394242', 'request_id': 'req-79b069ee-581e-474d-b24d-8d8e888e3d59'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:00:37.956033 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-79b069ee-581e-474d-b24d-8d8e888e3d59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36d0110a9d724746835cb719d0394242', 'request_id': 'req-79b069ee-581e-474d-b24d-8d8e888e3d59'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:00:37.966970 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-79b069ee-581e-474d-b24d-8d8e888e3d59 admin admin] The token cache is being invalidate because project 36d0110a9d724746835cb719d0394242 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 20:00:37.967257 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-79b069ee-581e-474d-b24d-8d8e888e3d59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:00:37.967951 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-79b069ee-581e-474d-b24d-8d8e888e3d59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:00:37.969468 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1323/2642] 10.222.0.61 () {58 vars in 1233 bytes} [Mon Aug 22 20:00:37 2022] DELETE /identity/v3/projects/36d0110a9d724746835cb719d0394242 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 20:00:37.973578 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba50121-8ddb-4267-809d-90615f51f00f service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:00:37.983638 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba50121-8ddb-4267-809d-90615f51f00f service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:00:37.993755 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:00:38.061765 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8500cb71-6f42-40a5-b678-4c57b57f3cb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:00:38.062410 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8500cb71-6f42-40a5-b678-4c57b57f3cb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:00:38.062688 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8500cb71-6f42-40a5-b678-4c57b57f3cb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:00:38.062953 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8500cb71-6f42-40a5-b678-4c57b57f3cb4 admin admin] PATH_INFO: `/v3/users/bd56880cb40c4727849680fbb2689188` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:00:38.064107 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8500cb71-6f42-40a5-b678-4c57b57f3cb4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd56880cb40c4727849680fbb2689188)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:00:38.066022 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8500cb71-6f42-40a5-b678-4c57b57f3cb4 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:00:38.114283 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-8500cb71-6f42-40a5-b678-4c57b57f3cb4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd56880cb40c4727849680fbb2689188'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:00:38.114973 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-8500cb71-6f42-40a5-b678-4c57b57f3cb4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd56880cb40c4727849680fbb2689188'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:00:38.122281 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-8500cb71-6f42-40a5-b678-4c57b57f3cb4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd56880cb40c4727849680fbb2689188'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:00:38.131631 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-8500cb71-6f42-40a5-b678-4c57b57f3cb4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd56880cb40c4727849680fbb2689188'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:00:38.145399 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-8500cb71-6f42-40a5-b678-4c57b57f3cb4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd56880cb40c4727849680fbb2689188'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:00:38.154574 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1320/2643] 10.222.0.61 () {58 vars in 1224 bytes} [Mon Aug 22 20:00:37 2022] DELETE /identity/v3/users/bd56880cb40c4727849680fbb2689188 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 20:00:38.440277 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b069ee-581e-474d-b24d-8d8e888e3d59 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:00:38.442071 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-79b069ee-581e-474d-b24d-8d8e888e3d59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:00:38.456785 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b069ee-581e-474d-b24d-8d8e888e3d59 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:00:38.516823 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7201c0-5d31-4c19-a84b-bfc316b6e7cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:00:38.517462 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7201c0-5d31-4c19-a84b-bfc316b6e7cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:00:38.517742 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7201c0-5d31-4c19-a84b-bfc316b6e7cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:00:38.518009 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7201c0-5d31-4c19-a84b-bfc316b6e7cf admin admin] PATH_INFO: `/v3/projects/a7136bdc811d44179225e35e80932926` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:00:38.519148 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7201c0-5d31-4c19-a84b-bfc316b6e7cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7136bdc811d44179225e35e80932926)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:00:38.521100 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7201c0-5d31-4c19-a84b-bfc316b6e7cf admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:00:38.557919 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:00:38.574210 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-8c7201c0-5d31-4c19-a84b-bfc316b6e7cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7136bdc811d44179225e35e80932926', 'request_id': 'req-8c7201c0-5d31-4c19-a84b-bfc316b6e7cf'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:00:38.584076 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-8c7201c0-5d31-4c19-a84b-bfc316b6e7cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7136bdc811d44179225e35e80932926', 'request_id': 'req-8c7201c0-5d31-4c19-a84b-bfc316b6e7cf'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:00:38.590898 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-8c7201c0-5d31-4c19-a84b-bfc316b6e7cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7136bdc811d44179225e35e80932926', 'request_id': 'req-8c7201c0-5d31-4c19-a84b-bfc316b6e7cf'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:00:38.601506 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-8c7201c0-5d31-4c19-a84b-bfc316b6e7cf admin admin] The token cache is being invalidate because project a7136bdc811d44179225e35e80932926 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 20:00:38.601798 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-8c7201c0-5d31-4c19-a84b-bfc316b6e7cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:00:38.602461 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-8c7201c0-5d31-4c19-a84b-bfc316b6e7cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:00:38.604011 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1324/2644] 10.222.0.61 () {58 vars in 1233 bytes} [Mon Aug 22 20:00:38 2022] DELETE /identity/v3/projects/a7136bdc811d44179225e35e80932926 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 20:00:38.609081 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebf478e-b2bd-40ab-bf08-4efe13d541b5 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:00:38.609608 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebf478e-b2bd-40ab-bf08-4efe13d541b5 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:00:38.609712 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebf478e-b2bd-40ab-bf08-4efe13d541b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:00:38.634993 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-8ebf478e-b2bd-40ab-bf08-4efe13d541b5 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 20:00:38.636276 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-8ebf478e-b2bd-40ab-bf08-4efe13d541b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:00:38.707132 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1321/2645] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 20:00:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 20:00:38.714993 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eeb237-5931-4214-a87b-528f7f5d34a1 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:00:38.715267 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eeb237-5931-4214-a87b-528f7f5d34a1 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:00:38.715541 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eeb237-5931-4214-a87b-528f7f5d34a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:00:38.735972 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-b5eeb237-5931-4214-a87b-528f7f5d34a1 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 20:00:38.737198 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-b5eeb237-5931-4214-a87b-528f7f5d34a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:00:38.745203 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1325/2646] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 20:00:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 20:00:38.750035 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebf478e-b2bd-40ab-bf08-4efe13d541b5 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:00:38.759841 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebf478e-b2bd-40ab-bf08-4efe13d541b5 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:00:38.765103 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea7410e-0d38-4c5e-8fa5-b4ae23a2d1a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:00:38.765718 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea7410e-0d38-4c5e-8fa5-b4ae23a2d1a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:00:38.765972 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea7410e-0d38-4c5e-8fa5-b4ae23a2d1a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:00:38.766247 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea7410e-0d38-4c5e-8fa5-b4ae23a2d1a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:00:38.766887 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea7410e-0d38-4c5e-8fa5-b4ae23a2d1a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 20:00:38.767184 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea7410e-0d38-4c5e-8fa5-b4ae23a2d1a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:00:38.768822 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea7410e-0d38-4c5e-8fa5-b4ae23a2d1a0 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:00:38.781206 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1322/2647] 10.222.0.61 () {58 vars in 1153 bytes} [Mon Aug 22 20:00:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 20:00:38.785638 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eeb237-5931-4214-a87b-528f7f5d34a1 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:00:38.795258 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eeb237-5931-4214-a87b-528f7f5d34a1 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:00:38.800003 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdb803a-3cc9-4572-b0eb-c53c63dd712d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:00:38.800614 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdb803a-3cc9-4572-b0eb-c53c63dd712d admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:00:38.800889 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdb803a-3cc9-4572-b0eb-c53c63dd712d admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:00:38.801160 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdb803a-3cc9-4572-b0eb-c53c63dd712d admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:00:38.801786 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdb803a-3cc9-4572-b0eb-c53c63dd712d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:00:38.803715 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdb803a-3cc9-4572-b0eb-c53c63dd712d admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:00:38.832783 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1326/2648] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 20:00:38 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 20:00:38.837434 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea7410e-0d38-4c5e-8fa5-b4ae23a2d1a0 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:00:38.847881 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea7410e-0d38-4c5e-8fa5-b4ae23a2d1a0 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:00:38.989434 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:00:39.038247 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ef3df1-a615-4781-a929-4efd21789bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:00:39.038867 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ef3df1-a615-4781-a929-4efd21789bcd admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:00:39.039144 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ef3df1-a615-4781-a929-4efd21789bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:00:39.039416 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ef3df1-a615-4781-a929-4efd21789bcd admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:00:39.040275 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ef3df1-a615-4781-a929-4efd21789bcd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:00:39.042377 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ef3df1-a615-4781-a929-4efd21789bcd admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:00:39.065141 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1323/2649] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 20:00:38 2022] POST /identity/v3/users => generated 293 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 20:00:39.069300 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdb803a-3cc9-4572-b0eb-c53c63dd712d admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:00:39.081288 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdb803a-3cc9-4572-b0eb-c53c63dd712d admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:00:39.085848 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d9cae3-b03d-4ca3-8737-fd2c3cb2b951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:00:39.086457 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d9cae3-b03d-4ca3-8737-fd2c3cb2b951 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:00:39.086732 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d9cae3-b03d-4ca3-8737-fd2c3cb2b951 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:00:39.087007 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d9cae3-b03d-4ca3-8737-fd2c3cb2b951 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:00:39.087660 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d9cae3-b03d-4ca3-8737-fd2c3cb2b951 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:00:39.089288 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d9cae3-b03d-4ca3-8737-fd2c3cb2b951 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:00:39.100391 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1327/2650] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 20:00:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 20:00:39.104544 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ef3df1-a615-4781-a929-4efd21789bcd admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:00:39.115631 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ef3df1-a615-4781-a929-4efd21789bcd admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:00:39.120393 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f12637-e936-4b6a-8c45-cc2012cb402a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:00:39.121001 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f12637-e936-4b6a-8c45-cc2012cb402a admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:00:39.121278 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f12637-e936-4b6a-8c45-cc2012cb402a admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:00:39.121554 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f12637-e936-4b6a-8c45-cc2012cb402a admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:00:39.122201 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f12637-e936-4b6a-8c45-cc2012cb402a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:00:39.123875 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f12637-e936-4b6a-8c45-cc2012cb402a admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:00:39.135382 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1324/2651] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 20:00:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 20:00:39.140441 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d9cae3-b03d-4ca3-8737-fd2c3cb2b951 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:00:39.157719 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d9cae3-b03d-4ca3-8737-fd2c3cb2b951 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:00:39.163113 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45792d3b-4f29-42a9-bce6-ea3c0bd11261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:00:39.163809 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45792d3b-4f29-42a9-bce6-ea3c0bd11261 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:00:39.164085 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45792d3b-4f29-42a9-bce6-ea3c0bd11261 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:00:39.164355 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45792d3b-4f29-42a9-bce6-ea3c0bd11261 admin admin] PATH_INFO: `/v3/projects/17529893f3ac490484021ad084d2ef2b/users/3ea4b37efd6e4b2bbd72cd6c90671bb3/roles/74a40bc819cb42eea7e966a4b2a016f1` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:00:39.179812 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45792d3b-4f29-42a9-bce6-ea3c0bd11261 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17529893f3ac490484021ad084d2ef2b, user_id=3ea4b37efd6e4b2bbd72cd6c90671bb3, role_id=74a40bc819cb42eea7e966a4b2a016f1)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:00:39.182411 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45792d3b-4f29-42a9-bce6-ea3c0bd11261 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:00:39.196865 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1328/2652] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 20:00:39 2022] PUT /identity/v3/projects/17529893f3ac490484021ad084d2ef2b/users/3ea4b37efd6e4b2bbd72cd6c90671bb3/roles/74a40bc819cb42eea7e966a4b2a016f1 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 20:00:39.209595 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8dd0e2-7d92-484d-abfc-893765c53296 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:00:39.209880 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8dd0e2-7d92-484d-abfc-893765c53296 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:00:39.210283 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8dd0e2-7d92-484d-abfc-893765c53296 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:00:39.243164 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-1e8dd0e2-7d92-484d-abfc-893765c53296 None None] MFA Rules not processed for user `3ea4b37efd6e4b2bbd72cd6c90671bb3`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 20:00:39.244381 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-1e8dd0e2-7d92-484d-abfc-893765c53296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:00:39.311031 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1325/2653] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 20:00:39 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 20:00:39.315582 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45792d3b-4f29-42a9-bce6-ea3c0bd11261 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:00:39.380691 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45792d3b-4f29-42a9-bce6-ea3c0bd11261 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:00:39.385518 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0783bb-8609-49ca-8c00-c98f53b632d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:00:39.386132 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0783bb-8609-49ca-8c00-c98f53b632d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:00:39.386413 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0783bb-8609-49ca-8c00-c98f53b632d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:00:39.386690 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0783bb-8609-49ca-8c00-c98f53b632d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:00:39.387343 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0783bb-8609-49ca-8c00-c98f53b632d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:00:39.389878 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0783bb-8609-49ca-8c00-c98f53b632d5 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:00:39.421173 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1329/2654] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 20:00:39 2022] POST /identity/v3/projects => generated 387 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 20:00:39.422703 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8dd0e2-7d92-484d-abfc-893765c53296 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:00:39.436909 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8dd0e2-7d92-484d-abfc-893765c53296 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:00:39.496025 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b916e7-ecec-429d-a7dd-69af07c88679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:00:39.496890 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b916e7-ecec-429d-a7dd-69af07c88679 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:00:39.497310 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b916e7-ecec-429d-a7dd-69af07c88679 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:00:39.497736 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b916e7-ecec-429d-a7dd-69af07c88679 admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:00:39.499090 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b916e7-ecec-429d-a7dd-69af07c88679 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:00:39.508940 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b916e7-ecec-429d-a7dd-69af07c88679 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:00:39.524180 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1326/2655] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 20:00:39 2022] POST /identity/v3/users => generated 300 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 20:00:39.528382 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0783bb-8609-49ca-8c00-c98f53b632d5 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:00:39.543828 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0783bb-8609-49ca-8c00-c98f53b632d5 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:00:39.551427 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf06312-8cb8-4807-a6c4-3015e285676a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:00:39.552488 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf06312-8cb8-4807-a6c4-3015e285676a admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:00:39.552973 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf06312-8cb8-4807-a6c4-3015e285676a admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:00:39.553460 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf06312-8cb8-4807-a6c4-3015e285676a admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:00:39.554324 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf06312-8cb8-4807-a6c4-3015e285676a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:00:39.555994 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf06312-8cb8-4807-a6c4-3015e285676a admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:00:39.568635 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1330/2656] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 20:00:39 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 20:00:39.572861 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b916e7-ecec-429d-a7dd-69af07c88679 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:00:39.583833 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b916e7-ecec-429d-a7dd-69af07c88679 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:00:39.588464 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e93f91-6912-4137-be6b-e09bc3f9697e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:00:39.589118 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e93f91-6912-4137-be6b-e09bc3f9697e admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:00:39.589390 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e93f91-6912-4137-be6b-e09bc3f9697e admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:00:39.589657 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e93f91-6912-4137-be6b-e09bc3f9697e admin admin] PATH_INFO: `/v3/projects/7c66666165534d928fbc7beeb2ebf517/users/74f2378e4f7042b9b63716bc929227bd/roles/163c30c3b3c246beb849ce176f02d9da` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:00:39.605632 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e93f91-6912-4137-be6b-e09bc3f9697e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c66666165534d928fbc7beeb2ebf517, user_id=74f2378e4f7042b9b63716bc929227bd, role_id=163c30c3b3c246beb849ce176f02d9da)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:00:39.608442 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e93f91-6912-4137-be6b-e09bc3f9697e admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:00:39.610769 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:00:39.626903 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1327/2657] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 20:00:39 2022] PUT /identity/v3/projects/7c66666165534d928fbc7beeb2ebf517/users/74f2378e4f7042b9b63716bc929227bd/roles/163c30c3b3c246beb849ce176f02d9da => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 20:00:39.630975 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf06312-8cb8-4807-a6c4-3015e285676a admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:00:39.642149 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf06312-8cb8-4807-a6c4-3015e285676a admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:00:39.662758 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:00:39.709438 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffc0af8-1270-48ee-a9cd-0882dc7bbfee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:00:39.710318 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffc0af8-1270-48ee-a9cd-0882dc7bbfee admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:00:39.710697 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffc0af8-1270-48ee-a9cd-0882dc7bbfee admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:00:39.710972 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffc0af8-1270-48ee-a9cd-0882dc7bbfee admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:00:39.711628 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffc0af8-1270-48ee-a9cd-0882dc7bbfee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:00:39.713375 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffc0af8-1270-48ee-a9cd-0882dc7bbfee admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:00:39.728445 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1331/2658] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 20:00:39 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 20:00:39.733187 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e93f91-6912-4137-be6b-e09bc3f9697e admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:00:39.745731 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e93f91-6912-4137-be6b-e09bc3f9697e admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:00:39.755150 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a64f35e-b4f6-425d-8aab-50267fb392ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:00:39.755794 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a64f35e-b4f6-425d-8aab-50267fb392ea admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:00:39.756074 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a64f35e-b4f6-425d-8aab-50267fb392ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:00:39.756348 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a64f35e-b4f6-425d-8aab-50267fb392ea admin admin] PATH_INFO: `/v3/projects/7c66666165534d928fbc7beeb2ebf517/users/74f2378e4f7042b9b63716bc929227bd/roles/163c30c3b3c246beb849ce176f02d9da` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:00:39.758314 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a64f35e-b4f6-425d-8aab-50267fb392ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c66666165534d928fbc7beeb2ebf517, user_id=74f2378e4f7042b9b63716bc929227bd, role_id=163c30c3b3c246beb849ce176f02d9da)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:00:39.761028 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a64f35e-b4f6-425d-8aab-50267fb392ea admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:00:39.780482 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1328/2659] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 20:00:39 2022] PUT /identity/v3/projects/7c66666165534d928fbc7beeb2ebf517/users/74f2378e4f7042b9b63716bc929227bd/roles/163c30c3b3c246beb849ce176f02d9da => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 20:00:39.798524 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09960c8-4f12-4368-87f2-8e5d41dc1bd2 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:00:39.798888 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09960c8-4f12-4368-87f2-8e5d41dc1bd2 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:00:39.799089 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09960c8-4f12-4368-87f2-8e5d41dc1bd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:00:39.828624 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-d09960c8-4f12-4368-87f2-8e5d41dc1bd2 None None] MFA Rules not processed for user `74f2378e4f7042b9b63716bc929227bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 20:00:39.830130 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-d09960c8-4f12-4368-87f2-8e5d41dc1bd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:00:39.903374 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1332/2660] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 20:00:39 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 20:00:39.914235 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a64f35e-b4f6-425d-8aab-50267fb392ea admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:00:39.916007 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-9a64f35e-b4f6-425d-8aab-50267fb392ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:00:39.976285 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a64f35e-b4f6-425d-8aab-50267fb392ea admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:00:39.980063 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07348c0b-dd67-4a1b-86d1-07abc93bbaa4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e79498873d4ed29a0e19bbea9b9214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:00:39.980678 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07348c0b-dd67-4a1b-86d1-07abc93bbaa4 service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:00:39.980955 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07348c0b-dd67-4a1b-86d1-07abc93bbaa4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:00:39.981279 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07348c0b-dd67-4a1b-86d1-07abc93bbaa4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:00:39.991491 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07348c0b-dd67-4a1b-86d1-07abc93bbaa4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:00:39.992939 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07348c0b-dd67-4a1b-86d1-07abc93bbaa4 service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:00:40.003762 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1329/2661] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 20:00:39 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 20:00:40.650484 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09960c8-4f12-4368-87f2-8e5d41dc1bd2 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:00:40.716262 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09960c8-4f12-4368-87f2-8e5d41dc1bd2 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:00:40.720787 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea124ede-96de-4080-9b57-8f9b24adc55c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:00:40.721419 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea124ede-96de-4080-9b57-8f9b24adc55c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:00:40.721690 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea124ede-96de-4080-9b57-8f9b24adc55c admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:00:40.721958 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea124ede-96de-4080-9b57-8f9b24adc55c admin admin] PATH_INFO: `/v3/users/3ea4b37efd6e4b2bbd72cd6c90671bb3` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:00:40.723069 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea124ede-96de-4080-9b57-8f9b24adc55c admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ea4b37efd6e4b2bbd72cd6c90671bb3)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:00:40.725022 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea124ede-96de-4080-9b57-8f9b24adc55c admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:00:40.771971 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-ea124ede-96de-4080-9b57-8f9b24adc55c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ea4b37efd6e4b2bbd72cd6c90671bb3'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:00:40.772619 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-ea124ede-96de-4080-9b57-8f9b24adc55c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ea4b37efd6e4b2bbd72cd6c90671bb3'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:00:40.773855 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:00:40.781903 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-ea124ede-96de-4080-9b57-8f9b24adc55c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ea4b37efd6e4b2bbd72cd6c90671bb3'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:00:40.790149 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-ea124ede-96de-4080-9b57-8f9b24adc55c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ea4b37efd6e4b2bbd72cd6c90671bb3'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:00:40.804455 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-ea124ede-96de-4080-9b57-8f9b24adc55c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ea4b37efd6e4b2bbd72cd6c90671bb3'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:00:40.811068 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1333/2662] 10.222.0.61 () {58 vars in 1224 bytes} [Mon Aug 22 20:00:40 2022] DELETE /identity/v3/users/3ea4b37efd6e4b2bbd72cd6c90671bb3 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 20:00:40.815802 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cc3529-3642-4b61-bf9a-b4a5cb00b3d7 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:00:40.816421 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cc3529-3642-4b61-bf9a-b4a5cb00b3d7 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:00:40.816421 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cc3529-3642-4b61-bf9a-b4a5cb00b3d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:00:40.835445 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-54cc3529-3642-4b61-bf9a-b4a5cb00b3d7 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 20:00:40.836699 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-54cc3529-3642-4b61-bf9a-b4a5cb00b3d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:00:40.896351 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1330/2663] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 20:00:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 20:00:40.906857 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea124ede-96de-4080-9b57-8f9b24adc55c admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:00:40.908321 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-ea124ede-96de-4080-9b57-8f9b24adc55c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:00:40.977337 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea124ede-96de-4080-9b57-8f9b24adc55c admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:00:40.981146 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836c68c4-ae6c-4322-b9d4-f73eeeee4c32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:00:40.981760 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836c68c4-ae6c-4322-b9d4-f73eeeee4c32 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:00:40.982038 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836c68c4-ae6c-4322-b9d4-f73eeeee4c32 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:00:40.982310 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836c68c4-ae6c-4322-b9d4-f73eeeee4c32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:00:40.992099 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836c68c4-ae6c-4322-b9d4-f73eeeee4c32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:00:40.993530 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836c68c4-ae6c-4322-b9d4-f73eeeee4c32 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:00:41.003831 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1334/2664] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 20:00:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 20:00:41.284566 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cc3529-3642-4b61-bf9a-b4a5cb00b3d7 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:00:41.286122 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-54cc3529-3642-4b61-bf9a-b4a5cb00b3d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:00:41.300775 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cc3529-3642-4b61-bf9a-b4a5cb00b3d7 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:00:41.305417 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04751830-385d-4a45-9c0d-0d62b830829a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:00:41.306038 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04751830-385d-4a45-9c0d-0d62b830829a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:00:41.306310 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04751830-385d-4a45-9c0d-0d62b830829a admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:00:41.306579 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04751830-385d-4a45-9c0d-0d62b830829a admin admin] PATH_INFO: `/v3/projects/17529893f3ac490484021ad084d2ef2b` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:00:41.307772 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04751830-385d-4a45-9c0d-0d62b830829a admin admin] RBAC: Authorizing `identity:delete_project(project_id=17529893f3ac490484021ad084d2ef2b)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:00:41.309806 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04751830-385d-4a45-9c0d-0d62b830829a admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:00:41.365129 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-04751830-385d-4a45-9c0d-0d62b830829a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17529893f3ac490484021ad084d2ef2b', 'request_id': 'req-04751830-385d-4a45-9c0d-0d62b830829a'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:00:41.380756 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-04751830-385d-4a45-9c0d-0d62b830829a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17529893f3ac490484021ad084d2ef2b', 'request_id': 'req-04751830-385d-4a45-9c0d-0d62b830829a'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:00:41.387838 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-04751830-385d-4a45-9c0d-0d62b830829a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17529893f3ac490484021ad084d2ef2b', 'request_id': 'req-04751830-385d-4a45-9c0d-0d62b830829a'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:00:41.398291 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-04751830-385d-4a45-9c0d-0d62b830829a admin admin] The token cache is being invalidate because project 17529893f3ac490484021ad084d2ef2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 20:00:41.398577 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-04751830-385d-4a45-9c0d-0d62b830829a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:00:41.399224 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-04751830-385d-4a45-9c0d-0d62b830829a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:00:41.400723 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1331/2665] 10.222.0.61 () {58 vars in 1233 bytes} [Mon Aug 22 20:00:41 2022] DELETE /identity/v3/projects/17529893f3ac490484021ad084d2ef2b => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 20:00:41.404731 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836c68c4-ae6c-4322-b9d4-f73eeeee4c32 service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:00:41.414811 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836c68c4-ae6c-4322-b9d4-f73eeeee4c32 service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:00:41.485315 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3656a2f2-7f10-405f-97f3-78ad4d35d4ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:00:41.485952 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3656a2f2-7f10-405f-97f3-78ad4d35d4ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:00:41.486228 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3656a2f2-7f10-405f-97f3-78ad4d35d4ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:00:41.486495 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3656a2f2-7f10-405f-97f3-78ad4d35d4ef admin admin] PATH_INFO: `/v3/users/74f2378e4f7042b9b63716bc929227bd` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:00:41.487654 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3656a2f2-7f10-405f-97f3-78ad4d35d4ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=74f2378e4f7042b9b63716bc929227bd)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:00:41.489580 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3656a2f2-7f10-405f-97f3-78ad4d35d4ef admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:00:41.539524 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-3656a2f2-7f10-405f-97f3-78ad4d35d4ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74f2378e4f7042b9b63716bc929227bd'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:00:41.540206 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-3656a2f2-7f10-405f-97f3-78ad4d35d4ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74f2378e4f7042b9b63716bc929227bd'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:00:41.548139 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-3656a2f2-7f10-405f-97f3-78ad4d35d4ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74f2378e4f7042b9b63716bc929227bd'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:00:41.556088 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-3656a2f2-7f10-405f-97f3-78ad4d35d4ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74f2378e4f7042b9b63716bc929227bd'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:00:41.562770 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:00:41.571589 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-3656a2f2-7f10-405f-97f3-78ad4d35d4ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74f2378e4f7042b9b63716bc929227bd'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:00:41.578285 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1335/2666] 10.222.0.61 () {58 vars in 1224 bytes} [Mon Aug 22 20:00:41 2022] DELETE /identity/v3/users/74f2378e4f7042b9b63716bc929227bd => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 20:00:41.861991 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04751830-385d-4a45-9c0d-0d62b830829a admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:00:41.863437 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-04751830-385d-4a45-9c0d-0d62b830829a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:00:41.879984 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04751830-385d-4a45-9c0d-0d62b830829a admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:00:41.884056 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:INSERT=3,keystone:SELECT=90,keystone:DELETE=7 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:00:41.950159 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c1c1be-0408-4313-82c4-684a3b3bead9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:00:41.950722 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c1c1be-0408-4313-82c4-684a3b3bead9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:00:41.951591 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c1c1be-0408-4313-82c4-684a3b3bead9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:00:41.951923 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c1c1be-0408-4313-82c4-684a3b3bead9 admin admin] PATH_INFO: `/v3/projects/7c66666165534d928fbc7beeb2ebf517` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:00:41.953080 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c1c1be-0408-4313-82c4-684a3b3bead9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c66666165534d928fbc7beeb2ebf517)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:00:41.955238 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c1c1be-0408-4313-82c4-684a3b3bead9 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:00:42.004809 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-95c1c1be-0408-4313-82c4-684a3b3bead9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c66666165534d928fbc7beeb2ebf517', 'request_id': 'req-95c1c1be-0408-4313-82c4-684a3b3bead9'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:00:42.012806 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-95c1c1be-0408-4313-82c4-684a3b3bead9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c66666165534d928fbc7beeb2ebf517', 'request_id': 'req-95c1c1be-0408-4313-82c4-684a3b3bead9'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:00:42.021062 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-95c1c1be-0408-4313-82c4-684a3b3bead9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c66666165534d928fbc7beeb2ebf517', 'request_id': 'req-95c1c1be-0408-4313-82c4-684a3b3bead9'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:00:42.032902 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-95c1c1be-0408-4313-82c4-684a3b3bead9 admin admin] The token cache is being invalidate because project 7c66666165534d928fbc7beeb2ebf517 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 20:00:42.033369 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-95c1c1be-0408-4313-82c4-684a3b3bead9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:00:42.033709 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-95c1c1be-0408-4313-82c4-684a3b3bead9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:00:42.035200 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1332/2667] 10.222.0.61 () {58 vars in 1233 bytes} [Mon Aug 22 20:00:41 2022] DELETE /identity/v3/projects/7c66666165534d928fbc7beeb2ebf517 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 20:00:42.040224 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2242b643-f70b-4b9f-b1bd-03f07566be51 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:00:42.040736 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2242b643-f70b-4b9f-b1bd-03f07566be51 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:00:42.040837 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2242b643-f70b-4b9f-b1bd-03f07566be51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:00:42.061257 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-2242b643-f70b-4b9f-b1bd-03f07566be51 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 20:00:42.062489 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-2242b643-f70b-4b9f-b1bd-03f07566be51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:00:42.127264 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1336/2668] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 20:00:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 20:00:42.135262 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aef4e2c-ea03-4fa0-8aff-f67462d1e8e5 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:00:42.135538 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aef4e2c-ea03-4fa0-8aff-f67462d1e8e5 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:00:42.135863 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aef4e2c-ea03-4fa0-8aff-f67462d1e8e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:00:42.157163 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-7aef4e2c-ea03-4fa0-8aff-f67462d1e8e5 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 20:00:42.158378 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-7aef4e2c-ea03-4fa0-8aff-f67462d1e8e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:00:42.166576 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1333/2669] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 20:00:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 20:00:42.171170 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2242b643-f70b-4b9f-b1bd-03f07566be51 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:00:42.182038 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2242b643-f70b-4b9f-b1bd-03f07566be51 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:00:42.186606 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818bd02c-3dcb-4bf4-9a80-aa3774118449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:00:42.187210 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818bd02c-3dcb-4bf4-9a80-aa3774118449 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:00:42.187489 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818bd02c-3dcb-4bf4-9a80-aa3774118449 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:00:42.187795 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818bd02c-3dcb-4bf4-9a80-aa3774118449 admin admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:00:42.188481 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818bd02c-3dcb-4bf4-9a80-aa3774118449 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 20:00:42.188775 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818bd02c-3dcb-4bf4-9a80-aa3774118449 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:00:42.190391 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818bd02c-3dcb-4bf4-9a80-aa3774118449 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:00:42.207906 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1337/2670] 10.222.0.61 () {58 vars in 1153 bytes} [Mon Aug 22 20:00:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 20:00:42.212197 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aef4e2c-ea03-4fa0-8aff-f67462d1e8e5 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:00:42.222762 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aef4e2c-ea03-4fa0-8aff-f67462d1e8e5 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:00:42.227635 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c37558-9186-4c94-a5bb-45d2d42db88b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:00:42.228264 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c37558-9186-4c94-a5bb-45d2d42db88b admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:00:42.228536 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c37558-9186-4c94-a5bb-45d2d42db88b admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:00:42.228803 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c37558-9186-4c94-a5bb-45d2d42db88b admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:00:42.229427 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c37558-9186-4c94-a5bb-45d2d42db88b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:00:42.231309 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c37558-9186-4c94-a5bb-45d2d42db88b admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:00:42.258777 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1334/2671] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 20:00:42 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 20:00:42.263167 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818bd02c-3dcb-4bf4-9a80-aa3774118449 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:00:42.274167 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818bd02c-3dcb-4bf4-9a80-aa3774118449 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:00:42.333775 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65983c05-4c1e-48e3-a746-a33daedc8e43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:00:42.334395 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65983c05-4c1e-48e3-a746-a33daedc8e43 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:00:42.334671 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65983c05-4c1e-48e3-a746-a33daedc8e43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:00:42.334949 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65983c05-4c1e-48e3-a746-a33daedc8e43 admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:00:42.335576 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65983c05-4c1e-48e3-a746-a33daedc8e43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:00:42.337524 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65983c05-4c1e-48e3-a746-a33daedc8e43 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:00:42.359818 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1338/2672] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 20:00:42 2022] POST /identity/v3/users => generated 286 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 20:00:42.364194 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c37558-9186-4c94-a5bb-45d2d42db88b admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:00:42.376278 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c37558-9186-4c94-a5bb-45d2d42db88b admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:00:42.380887 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361a0685-eb94-4272-b53e-dab15e43d746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:00:42.381491 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361a0685-eb94-4272-b53e-dab15e43d746 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:00:42.381782 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361a0685-eb94-4272-b53e-dab15e43d746 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:00:42.382036 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361a0685-eb94-4272-b53e-dab15e43d746 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:00:42.382681 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361a0685-eb94-4272-b53e-dab15e43d746 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:00:42.384318 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361a0685-eb94-4272-b53e-dab15e43d746 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:00:42.395349 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1335/2673] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 20:00:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 20:00:42.399539 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65983c05-4c1e-48e3-a746-a33daedc8e43 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:00:42.410883 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65983c05-4c1e-48e3-a746-a33daedc8e43 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:00:42.415980 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef105a2b-1d83-4466-8d94-ccffefe44731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:00:42.416676 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef105a2b-1d83-4466-8d94-ccffefe44731 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:00:42.416960 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef105a2b-1d83-4466-8d94-ccffefe44731 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:00:42.417238 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef105a2b-1d83-4466-8d94-ccffefe44731 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:00:42.417892 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef105a2b-1d83-4466-8d94-ccffefe44731 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:00:42.419513 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef105a2b-1d83-4466-8d94-ccffefe44731 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:00:42.431318 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1339/2674] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 20:00:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 20:00:42.436023 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361a0685-eb94-4272-b53e-dab15e43d746 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:00:42.455686 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361a0685-eb94-4272-b53e-dab15e43d746 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:00:42.460564 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f927ff15-4af2-47c5-8223-8af13185a663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:00:42.461172 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f927ff15-4af2-47c5-8223-8af13185a663 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:00:42.461470 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f927ff15-4af2-47c5-8223-8af13185a663 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:00:42.461740 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f927ff15-4af2-47c5-8223-8af13185a663 admin admin] PATH_INFO: `/v3/projects/88930697798d41cab0140fdb69f956f8/users/54675d93a5574390a951adcaeba7a0e9/roles/74a40bc819cb42eea7e966a4b2a016f1` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:00:42.477743 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f927ff15-4af2-47c5-8223-8af13185a663 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88930697798d41cab0140fdb69f956f8, user_id=54675d93a5574390a951adcaeba7a0e9, role_id=74a40bc819cb42eea7e966a4b2a016f1)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:00:42.480437 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f927ff15-4af2-47c5-8223-8af13185a663 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:00:42.494790 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1336/2675] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 20:00:42 2022] PUT /identity/v3/projects/88930697798d41cab0140fdb69f956f8/users/54675d93a5574390a951adcaeba7a0e9/roles/74a40bc819cb42eea7e966a4b2a016f1 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 20:00:42.500858 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c639f8f9-b6b4-4ce7-ac4e-72013fd93e67 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:00:42.501151 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c639f8f9-b6b4-4ce7-ac4e-72013fd93e67 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:00:42.501476 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c639f8f9-b6b4-4ce7-ac4e-72013fd93e67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:00:42.521189 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-c639f8f9-b6b4-4ce7-ac4e-72013fd93e67 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 20:00:42.522442 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-c639f8f9-b6b4-4ce7-ac4e-72013fd93e67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:00:42.571695 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:00:42.587393 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1340/2676] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 20:00:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 20:00:42.597078 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f927ff15-4af2-47c5-8223-8af13185a663 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:00:42.598840 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-f927ff15-4af2-47c5-8223-8af13185a663 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:00:42.640665 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:00:42.664164 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f927ff15-4af2-47c5-8223-8af13185a663 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:00:42.667851 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61671630-4325-4984-85bb-eaa50de79160 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:00:42.668460 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61671630-4325-4984-85bb-eaa50de79160 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:00:42.668736 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61671630-4325-4984-85bb-eaa50de79160 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:00:42.669004 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61671630-4325-4984-85bb-eaa50de79160 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:00:42.679546 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61671630-4325-4984-85bb-eaa50de79160 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:00:42.681134 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61671630-4325-4984-85bb-eaa50de79160 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:00:42.691735 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1337/2677] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 20:00:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 20:00:43.359197 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256bfacd-9bdc-4df3-8ceb-36ccb1583f1b None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:00:43.359776 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256bfacd-9bdc-4df3-8ceb-36ccb1583f1b None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:00:43.359986 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256bfacd-9bdc-4df3-8ceb-36ccb1583f1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:00:43.392349 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-256bfacd-9bdc-4df3-8ceb-36ccb1583f1b None None] MFA Rules not processed for user `54675d93a5574390a951adcaeba7a0e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 20:00:43.393754 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-256bfacd-9bdc-4df3-8ceb-36ccb1583f1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:00:43.447178 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61671630-4325-4984-85bb-eaa50de79160 service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:00:43.448789 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-61671630-4325-4984-85bb-eaa50de79160 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:00:43.478235 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1341/2678] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 20:00:43 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 20:00:43.482669 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256bfacd-9bdc-4df3-8ceb-36ccb1583f1b None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:00:43.495759 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256bfacd-9bdc-4df3-8ceb-36ccb1583f1b None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:00:43.501478 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa54c8e-e68c-4cc6-969f-a59112b0c8a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:00:43.502109 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa54c8e-e68c-4cc6-969f-a59112b0c8a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:00:43.502394 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa54c8e-e68c-4cc6-969f-a59112b0c8a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:00:43.502693 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa54c8e-e68c-4cc6-969f-a59112b0c8a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:00:43.503341 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa54c8e-e68c-4cc6-969f-a59112b0c8a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:00:43.505422 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa54c8e-e68c-4cc6-969f-a59112b0c8a1 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:00:43.528837 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61671630-4325-4984-85bb-eaa50de79160 service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:00:43.535968 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceb4822-b961-4a72-9b53-2c391cc2a9fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e79498873d4ed29a0e19bbea9b9214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:00:43.536603 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceb4822-b961-4a72-9b53-2c391cc2a9fd service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:00:43.536889 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceb4822-b961-4a72-9b53-2c391cc2a9fd service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:00:43.537158 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceb4822-b961-4a72-9b53-2c391cc2a9fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:00:43.538984 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-2ceb4822-b961-4a72-9b53-2c391cc2a9fd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:00:43.542636 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1342/2679] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 20:00:43 2022] POST /identity/v3/projects => generated 369 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 20:00:43.546027 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa54c8e-e68c-4cc6-969f-a59112b0c8a1 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:00:43.627038 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa54c8e-e68c-4cc6-969f-a59112b0c8a1 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:00:43.628639 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ceb4822-b961-4a72-9b53-2c391cc2a9fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:00:43.630416 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ceb4822-b961-4a72-9b53-2c391cc2a9fd service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:00:43.631940 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8b4201-5c8b-40ab-bb59-10063f704f51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:00:43.632566 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8b4201-5c8b-40ab-bb59-10063f704f51 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:00:43.632848 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8b4201-5c8b-40ab-bb59-10063f704f51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:00:43.633120 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8b4201-5c8b-40ab-bb59-10063f704f51 admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:00:43.633761 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8b4201-5c8b-40ab-bb59-10063f704f51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:00:43.635823 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8b4201-5c8b-40ab-bb59-10063f704f51 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:00:43.653211 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1338/2680] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 20:00:43 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 20:00:43.666612 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1343/2681] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 20:00:43 2022] POST /identity/v3/users => generated 291 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 20:00:43.667895 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceb4822-b961-4a72-9b53-2c391cc2a9fd service cinder] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:00:43.681048 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceb4822-b961-4a72-9b53-2c391cc2a9fd service cinder] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:00:43.688570 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e81bf5-0d12-44c6-89d7-d1f643f3e290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:00:43.689531 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e81bf5-0d12-44c6-89d7-d1f643f3e290 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:00:43.689995 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e81bf5-0d12-44c6-89d7-d1f643f3e290 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:00:43.690457 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e81bf5-0d12-44c6-89d7-d1f643f3e290 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:00:43.691484 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e81bf5-0d12-44c6-89d7-d1f643f3e290 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:00:43.694467 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e81bf5-0d12-44c6-89d7-d1f643f3e290 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:00:43.706881 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1339/2682] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 20:00:43 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 20:00:43.711064 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8b4201-5c8b-40ab-bb59-10063f704f51 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:00:43.721646 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8b4201-5c8b-40ab-bb59-10063f704f51 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:00:43.726377 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10f4192-1de3-42d8-b0b6-82d9ba4a9a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:00:43.727031 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10f4192-1de3-42d8-b0b6-82d9ba4a9a50 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:00:43.727310 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10f4192-1de3-42d8-b0b6-82d9ba4a9a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:00:43.727578 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10f4192-1de3-42d8-b0b6-82d9ba4a9a50 admin admin] PATH_INFO: `/v3/projects/146b2edc0a994259815742ab29a0c98d/users/96f2fee329f34708a32fa252287b6122/roles/163c30c3b3c246beb849ce176f02d9da` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:00:43.743440 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10f4192-1de3-42d8-b0b6-82d9ba4a9a50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=146b2edc0a994259815742ab29a0c98d, user_id=96f2fee329f34708a32fa252287b6122, role_id=163c30c3b3c246beb849ce176f02d9da)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:00:43.746142 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10f4192-1de3-42d8-b0b6-82d9ba4a9a50 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:00:43.767951 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1344/2683] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 20:00:43 2022] PUT /identity/v3/projects/146b2edc0a994259815742ab29a0c98d/users/96f2fee329f34708a32fa252287b6122/roles/163c30c3b3c246beb849ce176f02d9da => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 20:00:43.771981 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e81bf5-0d12-44c6-89d7-d1f643f3e290 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:00:43.784622 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e81bf5-0d12-44c6-89d7-d1f643f3e290 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:00:43.845490 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe91b82-ebde-4dd6-bc8b-5633d7b36eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:00:43.846106 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe91b82-ebde-4dd6-bc8b-5633d7b36eb8 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:00:43.846380 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe91b82-ebde-4dd6-bc8b-5633d7b36eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:00:43.846650 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe91b82-ebde-4dd6-bc8b-5633d7b36eb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:00:43.847290 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe91b82-ebde-4dd6-bc8b-5633d7b36eb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:00:43.849078 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe91b82-ebde-4dd6-bc8b-5633d7b36eb8 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:00:43.851461 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:00:43.861048 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1340/2684] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 20:00:43 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 20:00:43.865317 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10f4192-1de3-42d8-b0b6-82d9ba4a9a50 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:00:43.875704 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10f4192-1de3-42d8-b0b6-82d9ba4a9a50 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:00:43.880293 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbedd800-7ce2-437c-8a8b-c56e3b76bdfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:00:43.880904 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbedd800-7ce2-437c-8a8b-c56e3b76bdfc admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:00:43.881180 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbedd800-7ce2-437c-8a8b-c56e3b76bdfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:00:43.881450 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbedd800-7ce2-437c-8a8b-c56e3b76bdfc admin admin] PATH_INFO: `/v3/projects/146b2edc0a994259815742ab29a0c98d/users/96f2fee329f34708a32fa252287b6122/roles/163c30c3b3c246beb849ce176f02d9da` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:00:43.883394 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbedd800-7ce2-437c-8a8b-c56e3b76bdfc admin admin] RBAC: Authorizing `identity:create_grant(project_id=146b2edc0a994259815742ab29a0c98d, user_id=96f2fee329f34708a32fa252287b6122, role_id=163c30c3b3c246beb849ce176f02d9da)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:00:43.886065 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbedd800-7ce2-437c-8a8b-c56e3b76bdfc admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:00:43.900032 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1345/2685] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 20:00:43 2022] PUT /identity/v3/projects/146b2edc0a994259815742ab29a0c98d/users/96f2fee329f34708a32fa252287b6122/roles/163c30c3b3c246beb849ce176f02d9da => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 20:00:44.556176 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7265d7-d5ed-494d-b736-9263ae649e6a None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:00:44.557016 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7265d7-d5ed-494d-b736-9263ae649e6a None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:00:44.557016 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7265d7-d5ed-494d-b736-9263ae649e6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:00:44.592387 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-cd7265d7-d5ed-494d-b736-9263ae649e6a None None] MFA Rules not processed for user `96f2fee329f34708a32fa252287b6122`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 20:00:44.594325 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-cd7265d7-d5ed-494d-b736-9263ae649e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:00:44.681848 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1341/2686] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 20:00:44 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 20:00:44.693553 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbedd800-7ce2-437c-8a8b-c56e3b76bdfc admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:00:44.695298 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-cbedd800-7ce2-437c-8a8b-c56e3b76bdfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:00:44.711582 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:00:44.785562 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbedd800-7ce2-437c-8a8b-c56e3b76bdfc admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:00:44.789224 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2410daaa-a83d-4dc5-a323-52d4657ec8c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e79498873d4ed29a0e19bbea9b9214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:00:44.789849 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2410daaa-a83d-4dc5-a323-52d4657ec8c2 service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:00:44.790129 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2410daaa-a83d-4dc5-a323-52d4657ec8c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:00:44.790402 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2410daaa-a83d-4dc5-a323-52d4657ec8c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:00:44.863285 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2410daaa-a83d-4dc5-a323-52d4657ec8c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:00:44.864801 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2410daaa-a83d-4dc5-a323-52d4657ec8c2 service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:00:44.875183 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1346/2687] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 20:00:44 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 20:00:46.247190 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7265d7-d5ed-494d-b736-9263ae649e6a None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:00:46.257679 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7265d7-d5ed-494d-b736-9263ae649e6a None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:00:46.261238 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e90e0df-e2e8-494d-b15c-847af661500a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e79498873d4ed29a0e19bbea9b9214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:00:46.261846 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e90e0df-e2e8-494d-b15c-847af661500a service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:00:46.262120 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e90e0df-e2e8-494d-b15c-847af661500a service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:00:46.262391 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e90e0df-e2e8-494d-b15c-847af661500a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:00:46.272808 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e90e0df-e2e8-494d-b15c-847af661500a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:00:46.274239 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e90e0df-e2e8-494d-b15c-847af661500a service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:00:46.284935 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1342/2688] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 20:00:46 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 20:00:54.870198 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=42 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:00:55.155073 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2410daaa-a83d-4dc5-a323-52d4657ec8c2 service cinder] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:00:55.156525 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-2410daaa-a83d-4dc5-a323-52d4657ec8c2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:00:55.218283 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2410daaa-a83d-4dc5-a323-52d4657ec8c2 service cinder] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:00:55.222040 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa5949b-9f6f-4667-8034-4d068f7e5dfb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:00:55.222654 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa5949b-9f6f-4667-8034-4d068f7e5dfb service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:00:55.222925 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa5949b-9f6f-4667-8034-4d068f7e5dfb service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:00:55.223195 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa5949b-9f6f-4667-8034-4d068f7e5dfb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:00:55.225036 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-8fa5949b-9f6f-4667-8034-4d068f7e5dfb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:00:55.235197 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa5949b-9f6f-4667-8034-4d068f7e5dfb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:00:55.236662 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa5949b-9f6f-4667-8034-4d068f7e5dfb service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:00:55.245095 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1347/2689] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 20:00:55 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 20:00:56.280257 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=41 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:00:59.090845 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e90e0df-e2e8-494d-b15c-847af661500a service cinder] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:00:59.092470 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-5e90e0df-e2e8-494d-b15c-847af661500a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:00:59.183429 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e90e0df-e2e8-494d-b15c-847af661500a service cinder] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:00:59.192505 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1d9768-c8c1-4d09-bbe5-e2e3d09848a0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eef0f73297e04d62bc22acb790fad6ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:00:59.193196 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1d9768-c8c1-4d09-bbe5-e2e3d09848a0 service placement] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:00:59.193502 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1d9768-c8c1-4d09-bbe5-e2e3d09848a0 service placement] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:00:59.193888 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1d9768-c8c1-4d09-bbe5-e2e3d09848a0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:00:59.195750 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-cb1d9768-c8c1-4d09-bbe5-e2e3d09848a0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:00:59.209121 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1d9768-c8c1-4d09-bbe5-e2e3d09848a0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:00:59.210723 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1d9768-c8c1-4d09-bbe5-e2e3d09848a0 service placement] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:00:59.222114 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1343/2690] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 20:00:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 20:00:59.720536 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa5949b-9f6f-4667-8034-4d068f7e5dfb service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:00:59.802135 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa5949b-9f6f-4667-8034-4d068f7e5dfb service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:00:59.806548 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cd39c5-2dfe-4d9d-9be2-c15fbdf6715a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:00:59.807187 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cd39c5-2dfe-4d9d-9be2-c15fbdf6715a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:00:59.807459 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cd39c5-2dfe-4d9d-9be2-c15fbdf6715a admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:00:59.807766 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cd39c5-2dfe-4d9d-9be2-c15fbdf6715a admin admin] PATH_INFO: `/v3/users/54675d93a5574390a951adcaeba7a0e9` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:00:59.808936 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cd39c5-2dfe-4d9d-9be2-c15fbdf6715a admin admin] RBAC: Authorizing `identity:delete_user(user_id=54675d93a5574390a951adcaeba7a0e9)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:00:59.811497 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cd39c5-2dfe-4d9d-9be2-c15fbdf6715a admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:00:59.860146 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-33cd39c5-2dfe-4d9d-9be2-c15fbdf6715a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54675d93a5574390a951adcaeba7a0e9'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:00:59.860835 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-33cd39c5-2dfe-4d9d-9be2-c15fbdf6715a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54675d93a5574390a951adcaeba7a0e9'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:00:59.867871 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-33cd39c5-2dfe-4d9d-9be2-c15fbdf6715a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54675d93a5574390a951adcaeba7a0e9'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:00:59.877584 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-33cd39c5-2dfe-4d9d-9be2-c15fbdf6715a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54675d93a5574390a951adcaeba7a0e9'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:00:59.890207 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-33cd39c5-2dfe-4d9d-9be2-c15fbdf6715a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54675d93a5574390a951adcaeba7a0e9'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:00:59.896361 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1348/2691] 10.222.0.61 () {58 vars in 1224 bytes} [Mon Aug 22 20:00:59 2022] DELETE /identity/v3/users/54675d93a5574390a951adcaeba7a0e9 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 20:01:00.074940 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1d9768-c8c1-4d09-bbe5-e2e3d09848a0 service placement] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:01:00.093517 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1d9768-c8c1-4d09-bbe5-e2e3d09848a0 service placement] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:01:00.155568 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ea52cb-9a54-4546-b68e-ffaec21d50ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:01:00.156240 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ea52cb-9a54-4546-b68e-ffaec21d50ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:01:00.156520 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ea52cb-9a54-4546-b68e-ffaec21d50ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:01:00.156795 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ea52cb-9a54-4546-b68e-ffaec21d50ce admin admin] PATH_INFO: `/v3/projects/88930697798d41cab0140fdb69f956f8` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:01:00.157945 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ea52cb-9a54-4546-b68e-ffaec21d50ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=88930697798d41cab0140fdb69f956f8)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:01:00.159892 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ea52cb-9a54-4546-b68e-ffaec21d50ce admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:01:00.214239 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-98ea52cb-9a54-4546-b68e-ffaec21d50ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88930697798d41cab0140fdb69f956f8', 'request_id': 'req-98ea52cb-9a54-4546-b68e-ffaec21d50ce'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:01:00.221467 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-98ea52cb-9a54-4546-b68e-ffaec21d50ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88930697798d41cab0140fdb69f956f8', 'request_id': 'req-98ea52cb-9a54-4546-b68e-ffaec21d50ce'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:01:00.228240 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-98ea52cb-9a54-4546-b68e-ffaec21d50ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88930697798d41cab0140fdb69f956f8', 'request_id': 'req-98ea52cb-9a54-4546-b68e-ffaec21d50ce'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:01:00.238981 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-98ea52cb-9a54-4546-b68e-ffaec21d50ce admin admin] The token cache is being invalidate because project 88930697798d41cab0140fdb69f956f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 20:01:00.239266 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-98ea52cb-9a54-4546-b68e-ffaec21d50ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:01:00.239942 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-98ea52cb-9a54-4546-b68e-ffaec21d50ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:01:00.241405 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1344/2692] 10.222.0.61 () {58 vars in 1233 bytes} [Mon Aug 22 20:01:00 2022] DELETE /identity/v3/projects/88930697798d41cab0140fdb69f956f8 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 20:01:00.245509 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cd39c5-2dfe-4d9d-9be2-c15fbdf6715a admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:01:00.246967 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-33cd39c5-2dfe-4d9d-9be2-c15fbdf6715a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:01:00.260805 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cd39c5-2dfe-4d9d-9be2-c15fbdf6715a admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:01:00.286457 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:01:00.330588 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e05aa6-360b-4c29-ba51-104d569e32e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:01:00.331135 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e05aa6-360b-4c29-ba51-104d569e32e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:01:00.331411 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e05aa6-360b-4c29-ba51-104d569e32e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:01:00.331708 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e05aa6-360b-4c29-ba51-104d569e32e6 admin admin] PATH_INFO: `/v3/users/96f2fee329f34708a32fa252287b6122` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:01:00.332873 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e05aa6-360b-4c29-ba51-104d569e32e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=96f2fee329f34708a32fa252287b6122)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:01:00.334805 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e05aa6-360b-4c29-ba51-104d569e32e6 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:01:00.388132 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-87e05aa6-360b-4c29-ba51-104d569e32e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96f2fee329f34708a32fa252287b6122'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:01:00.388798 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-87e05aa6-360b-4c29-ba51-104d569e32e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96f2fee329f34708a32fa252287b6122'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:01:00.396018 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-87e05aa6-360b-4c29-ba51-104d569e32e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96f2fee329f34708a32fa252287b6122'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:01:00.410548 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-87e05aa6-360b-4c29-ba51-104d569e32e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96f2fee329f34708a32fa252287b6122'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:01:00.424957 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-87e05aa6-360b-4c29-ba51-104d569e32e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96f2fee329f34708a32fa252287b6122'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:01:00.431118 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1349/2693] 10.222.0.61 () {58 vars in 1224 bytes} [Mon Aug 22 20:01:00 2022] DELETE /identity/v3/users/96f2fee329f34708a32fa252287b6122 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 20:01:00.627724 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ea52cb-9a54-4546-b68e-ffaec21d50ce admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:01:00.629196 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-98ea52cb-9a54-4546-b68e-ffaec21d50ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:01:00.650716 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ea52cb-9a54-4546-b68e-ffaec21d50ce admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:01:00.671556 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:01:00.715515 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba3ffec-1c53-4f92-bfe3-7e76d8813cf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:01:00.716755 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba3ffec-1c53-4f92-bfe3-7e76d8813cf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:01:00.717036 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba3ffec-1c53-4f92-bfe3-7e76d8813cf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:01:00.717312 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba3ffec-1c53-4f92-bfe3-7e76d8813cf3 admin admin] PATH_INFO: `/v3/projects/146b2edc0a994259815742ab29a0c98d` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:01:00.718517 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba3ffec-1c53-4f92-bfe3-7e76d8813cf3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=146b2edc0a994259815742ab29a0c98d)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:01:00.720485 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba3ffec-1c53-4f92-bfe3-7e76d8813cf3 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:01:00.770003 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-eba3ffec-1c53-4f92-bfe3-7e76d8813cf3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '146b2edc0a994259815742ab29a0c98d', 'request_id': 'req-eba3ffec-1c53-4f92-bfe3-7e76d8813cf3'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:01:00.777563 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-eba3ffec-1c53-4f92-bfe3-7e76d8813cf3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '146b2edc0a994259815742ab29a0c98d', 'request_id': 'req-eba3ffec-1c53-4f92-bfe3-7e76d8813cf3'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:01:00.785061 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-eba3ffec-1c53-4f92-bfe3-7e76d8813cf3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '146b2edc0a994259815742ab29a0c98d', 'request_id': 'req-eba3ffec-1c53-4f92-bfe3-7e76d8813cf3'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:01:00.796666 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-eba3ffec-1c53-4f92-bfe3-7e76d8813cf3 admin admin] The token cache is being invalidate because project 146b2edc0a994259815742ab29a0c98d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 20:01:00.796957 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-eba3ffec-1c53-4f92-bfe3-7e76d8813cf3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:01:00.797652 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-eba3ffec-1c53-4f92-bfe3-7e76d8813cf3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:01:00.799185 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1345/2694] 10.222.0.61 () {58 vars in 1233 bytes} [Mon Aug 22 20:01:00 2022] DELETE /identity/v3/projects/146b2edc0a994259815742ab29a0c98d => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 20:01:00.804115 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90310fc-0524-4fcc-a89f-e1d78fb3011c None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:01:00.804642 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90310fc-0524-4fcc-a89f-e1d78fb3011c None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:01:00.804642 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90310fc-0524-4fcc-a89f-e1d78fb3011c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:01:00.824867 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-e90310fc-0524-4fcc-a89f-e1d78fb3011c None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 20:01:00.826104 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-e90310fc-0524-4fcc-a89f-e1d78fb3011c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:01:00.891518 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1350/2695] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 20:01:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 20:01:00.901150 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fd131c-a141-4cd6-8a1d-6955147d023e None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:01:00.901432 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fd131c-a141-4cd6-8a1d-6955147d023e None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:01:00.901702 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fd131c-a141-4cd6-8a1d-6955147d023e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:01:00.922383 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-95fd131c-a141-4cd6-8a1d-6955147d023e None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 20:01:00.923598 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-95fd131c-a141-4cd6-8a1d-6955147d023e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:01:00.931488 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1346/2696] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 20:01:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 20:01:00.935929 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90310fc-0524-4fcc-a89f-e1d78fb3011c None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:01:00.946595 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90310fc-0524-4fcc-a89f-e1d78fb3011c None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:01:00.951329 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b621afc2-ce8b-4f6b-9d56-984047a6790e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:01:00.951969 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b621afc2-ce8b-4f6b-9d56-984047a6790e admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:01:00.952248 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b621afc2-ce8b-4f6b-9d56-984047a6790e admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:01:00.952502 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b621afc2-ce8b-4f6b-9d56-984047a6790e admin admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:01:00.953149 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b621afc2-ce8b-4f6b-9d56-984047a6790e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 20:01:00.953444 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b621afc2-ce8b-4f6b-9d56-984047a6790e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:01:00.955321 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b621afc2-ce8b-4f6b-9d56-984047a6790e admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:01:00.967925 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1351/2697] 10.222.0.61 () {58 vars in 1153 bytes} [Mon Aug 22 20:01:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 20:01:00.972325 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fd131c-a141-4cd6-8a1d-6955147d023e None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:01:00.981522 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fd131c-a141-4cd6-8a1d-6955147d023e None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:01:00.987546 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7e2a29-2a66-4e1c-a016-df079135558e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:01:00.988284 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7e2a29-2a66-4e1c-a016-df079135558e admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:01:00.988658 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7e2a29-2a66-4e1c-a016-df079135558e admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:01:00.988931 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7e2a29-2a66-4e1c-a016-df079135558e admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:01:00.989570 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7e2a29-2a66-4e1c-a016-df079135558e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:01:00.991502 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7e2a29-2a66-4e1c-a016-df079135558e admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:01:01.020415 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1347/2698] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 20:01:00 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 20:01:01.024719 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b621afc2-ce8b-4f6b-9d56-984047a6790e admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:01:01.035602 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b621afc2-ce8b-4f6b-9d56-984047a6790e admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:01:01.062359 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:01:01.096833 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a74de5c-cbbb-4db3-a046-cea4420cfd57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:01:01.097475 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a74de5c-cbbb-4db3-a046-cea4420cfd57 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:01:01.097750 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a74de5c-cbbb-4db3-a046-cea4420cfd57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:01:01.098019 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a74de5c-cbbb-4db3-a046-cea4420cfd57 admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:01:01.098642 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a74de5c-cbbb-4db3-a046-cea4420cfd57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:01:01.100656 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a74de5c-cbbb-4db3-a046-cea4420cfd57 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:01:01.123220 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1352/2699] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 20:01:01 2022] POST /identity/v3/users => generated 287 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 20:01:01.127383 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7e2a29-2a66-4e1c-a016-df079135558e admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:01:01.138784 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7e2a29-2a66-4e1c-a016-df079135558e admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:01:01.143716 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c01906e-9200-4573-be16-5ef700ef36bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:01:01.144247 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c01906e-9200-4573-be16-5ef700ef36bb admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:01:01.144675 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c01906e-9200-4573-be16-5ef700ef36bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:01:01.144947 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c01906e-9200-4573-be16-5ef700ef36bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:01:01.145603 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c01906e-9200-4573-be16-5ef700ef36bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:01:01.147205 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c01906e-9200-4573-be16-5ef700ef36bb admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:01:01.158712 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1348/2700] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 20:01:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 20:01:01.162787 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a74de5c-cbbb-4db3-a046-cea4420cfd57 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:01:01.173425 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a74de5c-cbbb-4db3-a046-cea4420cfd57 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:01:01.178223 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2affb8-4e0b-4476-87b7-8c594deffb5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:01:01.178839 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2affb8-4e0b-4476-87b7-8c594deffb5d admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:01:01.179119 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2affb8-4e0b-4476-87b7-8c594deffb5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:01:01.179390 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2affb8-4e0b-4476-87b7-8c594deffb5d admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:01:01.180091 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2affb8-4e0b-4476-87b7-8c594deffb5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:01:01.181711 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2affb8-4e0b-4476-87b7-8c594deffb5d admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:01:01.193163 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1353/2701] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 20:01:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 20:01:01.197654 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c01906e-9200-4573-be16-5ef700ef36bb admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:01:01.208818 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c01906e-9200-4573-be16-5ef700ef36bb admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:01:01.213451 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4084ed9f-b37f-4e0d-8c95-dbebea09f887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:01:01.214105 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4084ed9f-b37f-4e0d-8c95-dbebea09f887 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:01:01.214377 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4084ed9f-b37f-4e0d-8c95-dbebea09f887 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:01:01.214643 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4084ed9f-b37f-4e0d-8c95-dbebea09f887 admin admin] PATH_INFO: `/v3/projects/ab1e27549f82409a9652af11c37d1c28/users/3fab47c7c17543379ed6616d851c64e8/roles/74a40bc819cb42eea7e966a4b2a016f1` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:01:01.230406 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4084ed9f-b37f-4e0d-8c95-dbebea09f887 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab1e27549f82409a9652af11c37d1c28, user_id=3fab47c7c17543379ed6616d851c64e8, role_id=74a40bc819cb42eea7e966a4b2a016f1)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:01:01.233607 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4084ed9f-b37f-4e0d-8c95-dbebea09f887 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:01:01.247066 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1349/2702] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 20:01:01 2022] PUT /identity/v3/projects/ab1e27549f82409a9652af11c37d1c28/users/3fab47c7c17543379ed6616d851c64e8/roles/74a40bc819cb42eea7e966a4b2a016f1 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 20:01:01.259034 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd9178b-70a8-4b97-a657-8ea865916ee0 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:01:01.259476 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd9178b-70a8-4b97-a657-8ea865916ee0 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:01:01.259978 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd9178b-70a8-4b97-a657-8ea865916ee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:01:01.292940 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-edd9178b-70a8-4b97-a657-8ea865916ee0 None None] MFA Rules not processed for user `3fab47c7c17543379ed6616d851c64e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 20:01:01.294172 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-edd9178b-70a8-4b97-a657-8ea865916ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:01:01.364643 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1354/2703] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 20:01:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 20:01:01.375888 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4084ed9f-b37f-4e0d-8c95-dbebea09f887 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:01:01.377719 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-4084ed9f-b37f-4e0d-8c95-dbebea09f887 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:01:01.432320 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:01:01.438545 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4084ed9f-b37f-4e0d-8c95-dbebea09f887 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:01:01.443960 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f046e69-ac7e-4d75-aabf-ec7105c14be1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e79498873d4ed29a0e19bbea9b9214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:01:01.444584 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f046e69-ac7e-4d75-aabf-ec7105c14be1 service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:01:01.445042 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f046e69-ac7e-4d75-aabf-ec7105c14be1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:01:01.445521 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f046e69-ac7e-4d75-aabf-ec7105c14be1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:01:01.455372 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f046e69-ac7e-4d75-aabf-ec7105c14be1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:01:01.456858 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f046e69-ac7e-4d75-aabf-ec7105c14be1 service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:01:01.467305 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1350/2704] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 20:01:01 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 20:01:01.488172 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd9178b-70a8-4b97-a657-8ea865916ee0 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:01:01.562923 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd9178b-70a8-4b97-a657-8ea865916ee0 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:01:01.567525 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692e16b0-21bb-42e4-9dee-e633c6ca94e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:01:01.568193 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692e16b0-21bb-42e4-9dee-e633c6ca94e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:01:01.568423 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692e16b0-21bb-42e4-9dee-e633c6ca94e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:01:01.568694 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692e16b0-21bb-42e4-9dee-e633c6ca94e8 admin admin] PATH_INFO: `/v3/users/3fab47c7c17543379ed6616d851c64e8` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:01:01.569828 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692e16b0-21bb-42e4-9dee-e633c6ca94e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3fab47c7c17543379ed6616d851c64e8)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:01:01.571772 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692e16b0-21bb-42e4-9dee-e633c6ca94e8 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:01:01.616472 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:01:01.625121 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-692e16b0-21bb-42e4-9dee-e633c6ca94e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3fab47c7c17543379ed6616d851c64e8'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:01:01.625778 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-692e16b0-21bb-42e4-9dee-e633c6ca94e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3fab47c7c17543379ed6616d851c64e8'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:01:01.633220 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-692e16b0-21bb-42e4-9dee-e633c6ca94e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3fab47c7c17543379ed6616d851c64e8'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:01:01.642724 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-692e16b0-21bb-42e4-9dee-e633c6ca94e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3fab47c7c17543379ed6616d851c64e8'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:01:01.656972 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-692e16b0-21bb-42e4-9dee-e633c6ca94e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3fab47c7c17543379ed6616d851c64e8'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:01:01.663448 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1355/2705] 10.222.0.61 () {58 vars in 1224 bytes} [Mon Aug 22 20:01:01 2022] DELETE /identity/v3/users/3fab47c7c17543379ed6616d851c64e8 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 20:01:01.668274 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3cf047-857f-4fd4-b30a-39f7345cc270 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:01:01.668560 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3cf047-857f-4fd4-b30a-39f7345cc270 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:01:01.668837 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3cf047-857f-4fd4-b30a-39f7345cc270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:01:01.689879 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-8e3cf047-857f-4fd4-b30a-39f7345cc270 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 20:01:01.691555 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-8e3cf047-857f-4fd4-b30a-39f7345cc270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:01:01.759387 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1351/2706] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 20:01:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 20:01:01.769157 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692e16b0-21bb-42e4-9dee-e633c6ca94e8 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:01:01.770607 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-692e16b0-21bb-42e4-9dee-e633c6ca94e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:01:01.833553 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692e16b0-21bb-42e4-9dee-e633c6ca94e8 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:01:01.837672 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec230870-7b07-431b-bf53-2f977d421e76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:01:01.838294 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec230870-7b07-431b-bf53-2f977d421e76 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:01:01.838569 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec230870-7b07-431b-bf53-2f977d421e76 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:01:01.838834 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec230870-7b07-431b-bf53-2f977d421e76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:01:01.850116 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec230870-7b07-431b-bf53-2f977d421e76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:01:01.851820 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec230870-7b07-431b-bf53-2f977d421e76 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:01:01.862315 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1356/2707] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 20:01:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 20:01:02.147366 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3cf047-857f-4fd4-b30a-39f7345cc270 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:01:02.148740 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-8e3cf047-857f-4fd4-b30a-39f7345cc270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:01:02.167389 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3cf047-857f-4fd4-b30a-39f7345cc270 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:01:02.172569 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b1d1e5-3365-4f65-a9b8-0d6edfd748c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:01:02.173395 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b1d1e5-3365-4f65-a9b8-0d6edfd748c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:01:02.173668 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b1d1e5-3365-4f65-a9b8-0d6edfd748c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:01:02.173933 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b1d1e5-3365-4f65-a9b8-0d6edfd748c5 admin admin] PATH_INFO: `/v3/projects/ab1e27549f82409a9652af11c37d1c28` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:01:02.175149 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b1d1e5-3365-4f65-a9b8-0d6edfd748c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab1e27549f82409a9652af11c37d1c28)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:01:02.177119 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b1d1e5-3365-4f65-a9b8-0d6edfd748c5 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:01:02.227827 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-63b1d1e5-3365-4f65-a9b8-0d6edfd748c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab1e27549f82409a9652af11c37d1c28', 'request_id': 'req-63b1d1e5-3365-4f65-a9b8-0d6edfd748c5'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:01:02.236625 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-63b1d1e5-3365-4f65-a9b8-0d6edfd748c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab1e27549f82409a9652af11c37d1c28', 'request_id': 'req-63b1d1e5-3365-4f65-a9b8-0d6edfd748c5'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:01:02.243576 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-63b1d1e5-3365-4f65-a9b8-0d6edfd748c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab1e27549f82409a9652af11c37d1c28', 'request_id': 'req-63b1d1e5-3365-4f65-a9b8-0d6edfd748c5'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:01:02.254895 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-63b1d1e5-3365-4f65-a9b8-0d6edfd748c5 admin admin] The token cache is being invalidate because project ab1e27549f82409a9652af11c37d1c28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 20:01:02.255182 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-63b1d1e5-3365-4f65-a9b8-0d6edfd748c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:01:02.255887 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-63b1d1e5-3365-4f65-a9b8-0d6edfd748c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:01:02.257699 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1352/2708] 10.222.0.61 () {58 vars in 1233 bytes} [Mon Aug 22 20:01:02 2022] DELETE /identity/v3/projects/ab1e27549f82409a9652af11c37d1c28 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 20:01:02.262629 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95084d2-5490-4d9d-9577-995b40b8f7ea None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:01:02.263549 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95084d2-5490-4d9d-9577-995b40b8f7ea None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:01:02.263944 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95084d2-5490-4d9d-9577-995b40b8f7ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:01:02.287480 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-d95084d2-5490-4d9d-9577-995b40b8f7ea None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 20:01:02.287480 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-d95084d2-5490-4d9d-9577-995b40b8f7ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:01:02.356476 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1357/2709] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 20:01:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 20:01:02.365020 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14d6296-e24d-44f4-8c30-23739c32a0da None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:01:02.365480 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14d6296-e24d-44f4-8c30-23739c32a0da None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:01:02.365928 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14d6296-e24d-44f4-8c30-23739c32a0da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:01:02.392169 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-e14d6296-e24d-44f4-8c30-23739c32a0da None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 20:01:02.393395 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-e14d6296-e24d-44f4-8c30-23739c32a0da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:01:02.401501 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1353/2710] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 20:01:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 20:01:02.406497 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95084d2-5490-4d9d-9577-995b40b8f7ea None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:01:02.416612 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95084d2-5490-4d9d-9577-995b40b8f7ea None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:01:02.421880 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363a198b-c227-4f07-9812-a294ae648591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:01:02.422497 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363a198b-c227-4f07-9812-a294ae648591 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:01:02.422772 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363a198b-c227-4f07-9812-a294ae648591 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:01:02.423049 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363a198b-c227-4f07-9812-a294ae648591 admin admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:01:02.423749 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363a198b-c227-4f07-9812-a294ae648591 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 20:01:02.424055 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363a198b-c227-4f07-9812-a294ae648591 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:01:02.425667 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363a198b-c227-4f07-9812-a294ae648591 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:01:02.438938 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1358/2711] 10.222.0.61 () {58 vars in 1153 bytes} [Mon Aug 22 20:01:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 20:01:02.442692 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14d6296-e24d-44f4-8c30-23739c32a0da None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:01:02.452546 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14d6296-e24d-44f4-8c30-23739c32a0da None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:01:02.457928 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a3b99d-87f2-4289-858d-de98c30de294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:01:02.458243 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a3b99d-87f2-4289-858d-de98c30de294 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:01:02.458523 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a3b99d-87f2-4289-858d-de98c30de294 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:01:02.458816 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a3b99d-87f2-4289-858d-de98c30de294 admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:01:02.459472 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a3b99d-87f2-4289-858d-de98c30de294 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:01:02.461445 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a3b99d-87f2-4289-858d-de98c30de294 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:01:02.492040 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1354/2712] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 20:01:02 2022] POST /identity/v3/projects => generated 361 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 20:01:02.496170 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363a198b-c227-4f07-9812-a294ae648591 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:01:02.507218 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363a198b-c227-4f07-9812-a294ae648591 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:01:02.552566 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:01:02.570521 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2693cc-a85c-474a-bfa1-2eba2ed31ae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:01:02.571442 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2693cc-a85c-474a-bfa1-2eba2ed31ae6 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:01:02.572004 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2693cc-a85c-474a-bfa1-2eba2ed31ae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:01:02.572411 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2693cc-a85c-474a-bfa1-2eba2ed31ae6 admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:01:02.573489 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2693cc-a85c-474a-bfa1-2eba2ed31ae6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:01:02.577433 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2693cc-a85c-474a-bfa1-2eba2ed31ae6 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:01:02.600209 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1359/2713] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 20:01:02 2022] POST /identity/v3/users => generated 281 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 20:01:02.604031 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a3b99d-87f2-4289-858d-de98c30de294 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:01:02.615462 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a3b99d-87f2-4289-858d-de98c30de294 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:01:02.620063 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b093987-8d9e-4180-9687-38872d93913b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:01:02.620690 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b093987-8d9e-4180-9687-38872d93913b admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:01:02.620962 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b093987-8d9e-4180-9687-38872d93913b admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:01:02.621253 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b093987-8d9e-4180-9687-38872d93913b admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:01:02.621892 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b093987-8d9e-4180-9687-38872d93913b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:01:02.624152 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b093987-8d9e-4180-9687-38872d93913b admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:01:02.634399 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1355/2714] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 20:01:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 20:01:02.638454 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2693cc-a85c-474a-bfa1-2eba2ed31ae6 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:01:02.649021 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2693cc-a85c-474a-bfa1-2eba2ed31ae6 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:01:02.653962 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b645ff-f128-464f-b507-d825bb388a54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:01:02.654629 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b645ff-f128-464f-b507-d825bb388a54 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:01:02.654909 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b645ff-f128-464f-b507-d825bb388a54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:01:02.655179 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b645ff-f128-464f-b507-d825bb388a54 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:01:02.655854 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b645ff-f128-464f-b507-d825bb388a54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:01:02.657461 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b645ff-f128-464f-b507-d825bb388a54 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:01:02.668531 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1360/2715] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 20:01:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 20:01:02.673548 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b093987-8d9e-4180-9687-38872d93913b admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:01:02.684143 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b093987-8d9e-4180-9687-38872d93913b admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:01:02.691761 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f251c00-d20c-4ddb-8971-e449b5bd979c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:01:02.693020 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f251c00-d20c-4ddb-8971-e449b5bd979c admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:01:02.693205 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f251c00-d20c-4ddb-8971-e449b5bd979c admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:01:02.694366 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f251c00-d20c-4ddb-8971-e449b5bd979c admin admin] PATH_INFO: `/v3/projects/8dd845bb622b499e8cae9440edb932b6/users/20ebf44d3b1b45b2a4596b3704a0a969/roles/74a40bc819cb42eea7e966a4b2a016f1` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:01:02.713420 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f251c00-d20c-4ddb-8971-e449b5bd979c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8dd845bb622b499e8cae9440edb932b6, user_id=20ebf44d3b1b45b2a4596b3704a0a969, role_id=74a40bc819cb42eea7e966a4b2a016f1)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:01:02.716175 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f251c00-d20c-4ddb-8971-e449b5bd979c admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:01:02.731337 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1356/2716] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 20:01:02 2022] PUT /identity/v3/projects/8dd845bb622b499e8cae9440edb932b6/users/20ebf44d3b1b45b2a4596b3704a0a969/roles/74a40bc819cb42eea7e966a4b2a016f1 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 20:01:02.743219 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d8e4b9-8011-4293-8b71-2e20c1726e3f None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:01:02.743424 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d8e4b9-8011-4293-8b71-2e20c1726e3f None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:01:02.743747 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d8e4b9-8011-4293-8b71-2e20c1726e3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:01:02.776089 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-35d8e4b9-8011-4293-8b71-2e20c1726e3f None None] MFA Rules not processed for user `20ebf44d3b1b45b2a4596b3704a0a969`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 20:01:02.777325 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-35d8e4b9-8011-4293-8b71-2e20c1726e3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:01:02.844167 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1361/2717] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 20:01:02 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 20:01:02.878771 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f251c00-d20c-4ddb-8971-e449b5bd979c admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:01:02.880519 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-0f251c00-d20c-4ddb-8971-e449b5bd979c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:01:02.891360 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:01:02.941232 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f251c00-d20c-4ddb-8971-e449b5bd979c admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:01:02.945070 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b66cb9-f811-4676-a293-31837101a054 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e79498873d4ed29a0e19bbea9b9214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:01:02.945679 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b66cb9-f811-4676-a293-31837101a054 service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:01:02.945961 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b66cb9-f811-4676-a293-31837101a054 service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:01:02.946232 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b66cb9-f811-4676-a293-31837101a054 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:01:02.955593 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b66cb9-f811-4676-a293-31837101a054 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:01:02.957090 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b66cb9-f811-4676-a293-31837101a054 service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:01:02.967521 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1357/2718] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 20:01:02 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 20:01:02.982474 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d8e4b9-8011-4293-8b71-2e20c1726e3f None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:01:03.047551 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d8e4b9-8011-4293-8b71-2e20c1726e3f None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:01:03.052334 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8a1709-c14d-45d1-8503-1ba376990ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:01:03.052982 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8a1709-c14d-45d1-8503-1ba376990ee3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:01:03.053259 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8a1709-c14d-45d1-8503-1ba376990ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:01:03.053531 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8a1709-c14d-45d1-8503-1ba376990ee3 admin admin] PATH_INFO: `/v3/users/20ebf44d3b1b45b2a4596b3704a0a969` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:01:03.054707 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8a1709-c14d-45d1-8503-1ba376990ee3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=20ebf44d3b1b45b2a4596b3704a0a969)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:01:03.056703 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8a1709-c14d-45d1-8503-1ba376990ee3 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:01:03.105020 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-9f8a1709-c14d-45d1-8503-1ba376990ee3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20ebf44d3b1b45b2a4596b3704a0a969'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:01:03.105668 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-9f8a1709-c14d-45d1-8503-1ba376990ee3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20ebf44d3b1b45b2a4596b3704a0a969'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:01:03.113467 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-9f8a1709-c14d-45d1-8503-1ba376990ee3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20ebf44d3b1b45b2a4596b3704a0a969'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:01:03.115217 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:01:03.121955 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-9f8a1709-c14d-45d1-8503-1ba376990ee3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20ebf44d3b1b45b2a4596b3704a0a969'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:01:03.135939 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-9f8a1709-c14d-45d1-8503-1ba376990ee3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20ebf44d3b1b45b2a4596b3704a0a969'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:01:03.142119 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1362/2719] 10.222.0.61 () {58 vars in 1224 bytes} [Mon Aug 22 20:01:02 2022] DELETE /identity/v3/users/20ebf44d3b1b45b2a4596b3704a0a969 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 20:01:03.146481 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11b5595-3b17-49de-807b-393909cf2d97 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:01:03.146755 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11b5595-3b17-49de-807b-393909cf2d97 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:01:03.147617 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11b5595-3b17-49de-807b-393909cf2d97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:01:03.166755 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-d11b5595-3b17-49de-807b-393909cf2d97 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 20:01:03.168024 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-d11b5595-3b17-49de-807b-393909cf2d97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:01:03.232531 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1358/2720] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 20:01:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 20:01:03.242386 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8a1709-c14d-45d1-8503-1ba376990ee3 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:01:03.243863 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-9f8a1709-c14d-45d1-8503-1ba376990ee3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:01:03.303861 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8a1709-c14d-45d1-8503-1ba376990ee3 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:01:03.307572 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8571c490-1e6a-4ae1-8ef3-e39396b0b651 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:01:03.308219 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8571c490-1e6a-4ae1-8ef3-e39396b0b651 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:01:03.308493 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8571c490-1e6a-4ae1-8ef3-e39396b0b651 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:01:03.308782 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8571c490-1e6a-4ae1-8ef3-e39396b0b651 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:01:03.318966 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8571c490-1e6a-4ae1-8ef3-e39396b0b651 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:01:03.321513 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8571c490-1e6a-4ae1-8ef3-e39396b0b651 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:01:03.332851 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1363/2721] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 20:01:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 20:01:03.627506 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11b5595-3b17-49de-807b-393909cf2d97 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:01:03.629026 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-d11b5595-3b17-49de-807b-393909cf2d97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:01:03.641294 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11b5595-3b17-49de-807b-393909cf2d97 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:01:03.646200 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1910ee-f93b-45be-b6dc-537d34c3a321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:01:03.646528 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1910ee-f93b-45be-b6dc-537d34c3a321 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:01:03.646800 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1910ee-f93b-45be-b6dc-537d34c3a321 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:01:03.647063 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1910ee-f93b-45be-b6dc-537d34c3a321 admin admin] PATH_INFO: `/v3/projects/8dd845bb622b499e8cae9440edb932b6` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:01:03.648236 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1910ee-f93b-45be-b6dc-537d34c3a321 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8dd845bb622b499e8cae9440edb932b6)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:01:03.650137 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1910ee-f93b-45be-b6dc-537d34c3a321 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:01:03.710004 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-0f1910ee-f93b-45be-b6dc-537d34c3a321 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8dd845bb622b499e8cae9440edb932b6', 'request_id': 'req-0f1910ee-f93b-45be-b6dc-537d34c3a321'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:01:03.713119 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-0f1910ee-f93b-45be-b6dc-537d34c3a321 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8dd845bb622b499e8cae9440edb932b6', 'request_id': 'req-0f1910ee-f93b-45be-b6dc-537d34c3a321'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:01:03.720579 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-0f1910ee-f93b-45be-b6dc-537d34c3a321 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8dd845bb622b499e8cae9440edb932b6', 'request_id': 'req-0f1910ee-f93b-45be-b6dc-537d34c3a321'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:01:03.731177 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-0f1910ee-f93b-45be-b6dc-537d34c3a321 admin admin] The token cache is being invalidate because project 8dd845bb622b499e8cae9440edb932b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 20:01:03.731488 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-0f1910ee-f93b-45be-b6dc-537d34c3a321 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:01:03.732768 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-0f1910ee-f93b-45be-b6dc-537d34c3a321 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:01:03.734583 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1359/2722] 10.222.0.61 () {58 vars in 1233 bytes} [Mon Aug 22 20:01:03 2022] DELETE /identity/v3/projects/8dd845bb622b499e8cae9440edb932b6 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 20:01:03.739493 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573b501a-7af4-4730-a24f-fd437d7fa24c None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:01:03.739855 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573b501a-7af4-4730-a24f-fd437d7fa24c None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:01:03.740087 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573b501a-7af4-4730-a24f-fd437d7fa24c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:01:03.767276 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-573b501a-7af4-4730-a24f-fd437d7fa24c None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 20:01:03.768558 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-573b501a-7af4-4730-a24f-fd437d7fa24c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:01:03.837211 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1364/2723] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 20:01:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 20:01:03.845381 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e218041a-b282-4332-8d2d-4dd89064665f None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:01:03.845679 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e218041a-b282-4332-8d2d-4dd89064665f None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:01:03.845939 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e218041a-b282-4332-8d2d-4dd89064665f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:01:03.865893 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-e218041a-b282-4332-8d2d-4dd89064665f None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 20:01:03.867131 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-e218041a-b282-4332-8d2d-4dd89064665f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:01:03.875107 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1360/2724] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 20:01:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 20:01:03.879387 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573b501a-7af4-4730-a24f-fd437d7fa24c None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:01:03.889655 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573b501a-7af4-4730-a24f-fd437d7fa24c None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:01:03.894443 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e999253c-3d96-4243-a803-082b28c23306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:01:03.895051 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e999253c-3d96-4243-a803-082b28c23306 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:01:03.895341 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e999253c-3d96-4243-a803-082b28c23306 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:01:03.895615 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e999253c-3d96-4243-a803-082b28c23306 admin admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:01:03.896308 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e999253c-3d96-4243-a803-082b28c23306 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 20:01:03.896615 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e999253c-3d96-4243-a803-082b28c23306 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:01:03.898228 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e999253c-3d96-4243-a803-082b28c23306 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:01:03.911279 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1365/2725] 10.222.0.61 () {58 vars in 1153 bytes} [Mon Aug 22 20:01:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 20:01:03.915738 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e218041a-b282-4332-8d2d-4dd89064665f None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:01:03.925944 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e218041a-b282-4332-8d2d-4dd89064665f None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:01:03.930851 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4dac3a-5c89-421e-91fd-624256c278b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:01:03.931457 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4dac3a-5c89-421e-91fd-624256c278b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:01:03.931792 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4dac3a-5c89-421e-91fd-624256c278b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:01:03.932112 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4dac3a-5c89-421e-91fd-624256c278b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:01:03.932699 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4dac3a-5c89-421e-91fd-624256c278b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:01:03.934596 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4dac3a-5c89-421e-91fd-624256c278b0 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:01:03.962801 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1361/2726] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 20:01:03 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 20:01:03.967672 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e999253c-3d96-4243-a803-082b28c23306 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:01:03.978199 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e999253c-3d96-4243-a803-082b28c23306 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:01:04.034671 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c979d70-46a9-4e74-af3c-6ebd32d440c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:01:04.035303 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c979d70-46a9-4e74-af3c-6ebd32d440c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:01:04.035622 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c979d70-46a9-4e74-af3c-6ebd32d440c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:01:04.035891 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c979d70-46a9-4e74-af3c-6ebd32d440c0 admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:01:04.036540 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c979d70-46a9-4e74-af3c-6ebd32d440c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:01:04.038564 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c979d70-46a9-4e74-af3c-6ebd32d440c0 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:01:04.049390 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:INSERT=3,keystone:DELETE=3,keystone:SELECT=94 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:01:04.062711 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1366/2727] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 20:01:03 2022] POST /identity/v3/users => generated 287 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 20:01:04.070418 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4dac3a-5c89-421e-91fd-624256c278b0 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:01:04.084077 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4dac3a-5c89-421e-91fd-624256c278b0 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:01:04.089788 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb73a6e6-3b21-4295-ba9a-eb02bf7d12ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:01:04.090314 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb73a6e6-3b21-4295-ba9a-eb02bf7d12ee admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:01:04.090591 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb73a6e6-3b21-4295-ba9a-eb02bf7d12ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:01:04.090862 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb73a6e6-3b21-4295-ba9a-eb02bf7d12ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:01:04.092124 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb73a6e6-3b21-4295-ba9a-eb02bf7d12ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:01:04.093726 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb73a6e6-3b21-4295-ba9a-eb02bf7d12ee admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:01:04.095184 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:01:04.105971 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1362/2728] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 20:01:04 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 20:01:04.113555 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c979d70-46a9-4e74-af3c-6ebd32d440c0 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:01:04.125227 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c979d70-46a9-4e74-af3c-6ebd32d440c0 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:01:04.130872 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e145c98-7b01-484b-a6cd-9af6109a28ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:01:04.131486 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e145c98-7b01-484b-a6cd-9af6109a28ec admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:01:04.131801 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e145c98-7b01-484b-a6cd-9af6109a28ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:01:04.132103 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e145c98-7b01-484b-a6cd-9af6109a28ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:01:04.132756 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e145c98-7b01-484b-a6cd-9af6109a28ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:01:04.134362 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e145c98-7b01-484b-a6cd-9af6109a28ec admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:01:04.146326 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1367/2729] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 20:01:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 20:01:04.150491 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb73a6e6-3b21-4295-ba9a-eb02bf7d12ee admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:01:04.161522 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb73a6e6-3b21-4295-ba9a-eb02bf7d12ee admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:01:04.166336 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaef3ab-2111-4a49-bee9-800268f9c8fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:01:04.166999 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaef3ab-2111-4a49-bee9-800268f9c8fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:01:04.167315 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaef3ab-2111-4a49-bee9-800268f9c8fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:01:04.167592 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaef3ab-2111-4a49-bee9-800268f9c8fe admin admin] PATH_INFO: `/v3/projects/bd5a652d8499488da159a2e004cdd49d/users/c240f14d16f44cdfaf6666ff543fed4e/roles/74a40bc819cb42eea7e966a4b2a016f1` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:01:04.183752 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaef3ab-2111-4a49-bee9-800268f9c8fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd5a652d8499488da159a2e004cdd49d, user_id=c240f14d16f44cdfaf6666ff543fed4e, role_id=74a40bc819cb42eea7e966a4b2a016f1)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:01:04.186498 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaef3ab-2111-4a49-bee9-800268f9c8fe admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:01:04.199995 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1363/2730] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 20:01:04 2022] PUT /identity/v3/projects/bd5a652d8499488da159a2e004cdd49d/users/c240f14d16f44cdfaf6666ff543fed4e/roles/74a40bc819cb42eea7e966a4b2a016f1 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 20:01:04.211446 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35335117-5ba6-4375-aecb-944031e65fc2 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:01:04.211779 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35335117-5ba6-4375-aecb-944031e65fc2 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:01:04.212055 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35335117-5ba6-4375-aecb-944031e65fc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:01:04.242246 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-35335117-5ba6-4375-aecb-944031e65fc2 None None] MFA Rules not processed for user `c240f14d16f44cdfaf6666ff543fed4e`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 20:01:04.243503 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-35335117-5ba6-4375-aecb-944031e65fc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:01:04.312020 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1368/2731] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 20:01:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 20:01:04.322899 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaef3ab-2111-4a49-bee9-800268f9c8fe admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:01:04.325820 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-4aaef3ab-2111-4a49-bee9-800268f9c8fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:01:04.389291 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaef3ab-2111-4a49-bee9-800268f9c8fe admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:01:04.393003 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20e0ea2-a700-468a-a9ba-8e0b24419dd2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e79498873d4ed29a0e19bbea9b9214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:01:04.393546 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20e0ea2-a700-468a-a9ba-8e0b24419dd2 service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:01:04.393821 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20e0ea2-a700-468a-a9ba-8e0b24419dd2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:01:04.394142 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20e0ea2-a700-468a-a9ba-8e0b24419dd2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:01:04.402995 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20e0ea2-a700-468a-a9ba-8e0b24419dd2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:01:04.405054 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20e0ea2-a700-468a-a9ba-8e0b24419dd2 service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:01:04.414089 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1364/2732] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 20:01:04 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 20:01:07.516443 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35335117-5ba6-4375-aecb-944031e65fc2 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:01:07.587532 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35335117-5ba6-4375-aecb-944031e65fc2 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:01:07.592638 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d812d78-167f-4d18-923b-6d2a17e39a14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:01:07.593275 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d812d78-167f-4d18-923b-6d2a17e39a14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:01:07.593555 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d812d78-167f-4d18-923b-6d2a17e39a14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:01:07.593826 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d812d78-167f-4d18-923b-6d2a17e39a14 admin admin] PATH_INFO: `/v3/users/c240f14d16f44cdfaf6666ff543fed4e` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:01:07.594959 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d812d78-167f-4d18-923b-6d2a17e39a14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c240f14d16f44cdfaf6666ff543fed4e)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:01:07.597081 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d812d78-167f-4d18-923b-6d2a17e39a14 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:01:07.658816 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-8d812d78-167f-4d18-923b-6d2a17e39a14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c240f14d16f44cdfaf6666ff543fed4e'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:01:07.659566 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-8d812d78-167f-4d18-923b-6d2a17e39a14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c240f14d16f44cdfaf6666ff543fed4e'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:01:07.667065 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-8d812d78-167f-4d18-923b-6d2a17e39a14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c240f14d16f44cdfaf6666ff543fed4e'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:01:07.675618 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-8d812d78-167f-4d18-923b-6d2a17e39a14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c240f14d16f44cdfaf6666ff543fed4e'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:01:07.684431 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:INSERT=2,keystone:SELECT=89,keystone:DELETE=9 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:01:07.690120 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-8d812d78-167f-4d18-923b-6d2a17e39a14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c240f14d16f44cdfaf6666ff543fed4e'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:01:07.697039 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1369/2733] 10.222.0.61 () {58 vars in 1224 bytes} [Mon Aug 22 20:01:07 2022] DELETE /identity/v3/users/c240f14d16f44cdfaf6666ff543fed4e => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 20:01:07.701595 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f59dce0-d31e-4bac-b3a1-feda4bfd170a None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:01:07.702091 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f59dce0-d31e-4bac-b3a1-feda4bfd170a None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:01:07.702184 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f59dce0-d31e-4bac-b3a1-feda4bfd170a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:01:07.723446 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-0f59dce0-d31e-4bac-b3a1-feda4bfd170a None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 20:01:07.724927 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-0f59dce0-d31e-4bac-b3a1-feda4bfd170a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:01:07.788022 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1365/2734] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 20:01:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 20:01:07.799428 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d812d78-167f-4d18-923b-6d2a17e39a14 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:01:07.800899 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-8d812d78-167f-4d18-923b-6d2a17e39a14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:01:07.861370 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d812d78-167f-4d18-923b-6d2a17e39a14 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:01:07.865154 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898367b0-4e91-40bb-b9a9-4ace92f3e886 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:01:07.865769 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898367b0-4e91-40bb-b9a9-4ace92f3e886 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:01:07.866048 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898367b0-4e91-40bb-b9a9-4ace92f3e886 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:01:07.866320 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898367b0-4e91-40bb-b9a9-4ace92f3e886 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:01:07.875778 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898367b0-4e91-40bb-b9a9-4ace92f3e886 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:01:07.877220 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898367b0-4e91-40bb-b9a9-4ace92f3e886 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:01:07.888770 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1370/2735] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 20:01:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 20:01:08.187828 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f59dce0-d31e-4bac-b3a1-feda4bfd170a None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:01:08.189768 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-0f59dce0-d31e-4bac-b3a1-feda4bfd170a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:01:08.205375 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f59dce0-d31e-4bac-b3a1-feda4bfd170a None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:01:08.211440 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e47ef29-ff90-4fbd-91eb-25fed93feaa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:01:08.212292 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e47ef29-ff90-4fbd-91eb-25fed93feaa6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:01:08.212639 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e47ef29-ff90-4fbd-91eb-25fed93feaa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:01:08.212978 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e47ef29-ff90-4fbd-91eb-25fed93feaa6 admin admin] PATH_INFO: `/v3/projects/bd5a652d8499488da159a2e004cdd49d` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:01:08.214198 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e47ef29-ff90-4fbd-91eb-25fed93feaa6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd5a652d8499488da159a2e004cdd49d)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:01:08.217978 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e47ef29-ff90-4fbd-91eb-25fed93feaa6 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:01:08.272806 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-6e47ef29-ff90-4fbd-91eb-25fed93feaa6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd5a652d8499488da159a2e004cdd49d', 'request_id': 'req-6e47ef29-ff90-4fbd-91eb-25fed93feaa6'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:01:08.284708 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-6e47ef29-ff90-4fbd-91eb-25fed93feaa6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd5a652d8499488da159a2e004cdd49d', 'request_id': 'req-6e47ef29-ff90-4fbd-91eb-25fed93feaa6'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:01:08.292375 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-6e47ef29-ff90-4fbd-91eb-25fed93feaa6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd5a652d8499488da159a2e004cdd49d', 'request_id': 'req-6e47ef29-ff90-4fbd-91eb-25fed93feaa6'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:01:08.299072 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:01:08.304205 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-6e47ef29-ff90-4fbd-91eb-25fed93feaa6 admin admin] The token cache is being invalidate because project bd5a652d8499488da159a2e004cdd49d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 20:01:08.304746 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-6e47ef29-ff90-4fbd-91eb-25fed93feaa6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:01:08.305787 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-6e47ef29-ff90-4fbd-91eb-25fed93feaa6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:01:08.307397 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1366/2736] 10.222.0.61 () {58 vars in 1233 bytes} [Mon Aug 22 20:01:08 2022] DELETE /identity/v3/projects/bd5a652d8499488da159a2e004cdd49d => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 20:01:08.313246 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09253daf-6ca5-4f9a-b0fb-77dc8d920ede None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:01:08.313530 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09253daf-6ca5-4f9a-b0fb-77dc8d920ede None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:01:08.313803 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09253daf-6ca5-4f9a-b0fb-77dc8d920ede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:01:08.343581 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-09253daf-6ca5-4f9a-b0fb-77dc8d920ede None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 20:01:08.344923 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-09253daf-6ca5-4f9a-b0fb-77dc8d920ede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:01:08.408026 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1371/2737] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 20:01:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 20:01:08.420904 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf55ac2-74e3-4a26-ab9e-443357f5b20e None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:01:08.421178 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf55ac2-74e3-4a26-ab9e-443357f5b20e None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:01:08.421445 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf55ac2-74e3-4a26-ab9e-443357f5b20e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:01:08.441777 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-dbf55ac2-74e3-4a26-ab9e-443357f5b20e None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 20:01:08.443004 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-dbf55ac2-74e3-4a26-ab9e-443357f5b20e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:01:08.450990 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1367/2738] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 20:01:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 20:01:08.455575 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09253daf-6ca5-4f9a-b0fb-77dc8d920ede None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:01:08.466928 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09253daf-6ca5-4f9a-b0fb-77dc8d920ede None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:01:08.471656 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6059cd09-f89f-47be-8d48-4af349f3d602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:01:08.472373 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6059cd09-f89f-47be-8d48-4af349f3d602 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:01:08.472738 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6059cd09-f89f-47be-8d48-4af349f3d602 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:01:08.473010 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6059cd09-f89f-47be-8d48-4af349f3d602 admin admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:01:08.473658 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6059cd09-f89f-47be-8d48-4af349f3d602 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 20:01:08.473956 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6059cd09-f89f-47be-8d48-4af349f3d602 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:01:08.476679 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6059cd09-f89f-47be-8d48-4af349f3d602 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:01:08.492414 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1372/2739] 10.222.0.61 () {58 vars in 1153 bytes} [Mon Aug 22 20:01:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 20:01:08.496849 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf55ac2-74e3-4a26-ab9e-443357f5b20e None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:01:08.506676 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf55ac2-74e3-4a26-ab9e-443357f5b20e None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:01:08.511680 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e144a5a1-bcbc-4642-a5d1-06c46a87546a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:01:08.512295 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e144a5a1-bcbc-4642-a5d1-06c46a87546a admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:01:08.512573 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e144a5a1-bcbc-4642-a5d1-06c46a87546a admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:01:08.512844 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e144a5a1-bcbc-4642-a5d1-06c46a87546a admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:01:08.513498 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e144a5a1-bcbc-4642-a5d1-06c46a87546a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:01:08.515397 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e144a5a1-bcbc-4642-a5d1-06c46a87546a admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:01:08.545418 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1368/2740] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 20:01:08 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 20:01:08.549890 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6059cd09-f89f-47be-8d48-4af349f3d602 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:01:08.560309 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6059cd09-f89f-47be-8d48-4af349f3d602 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:01:08.619585 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1211a92c-157e-421a-a5f8-5b5f422ff9d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:01:08.620231 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1211a92c-157e-421a-a5f8-5b5f422ff9d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:01:08.620502 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1211a92c-157e-421a-a5f8-5b5f422ff9d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:01:08.620768 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1211a92c-157e-421a-a5f8-5b5f422ff9d0 admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:01:08.621397 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1211a92c-157e-421a-a5f8-5b5f422ff9d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:01:08.623342 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1211a92c-157e-421a-a5f8-5b5f422ff9d0 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:01:08.648694 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1373/2741] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 20:01:08 2022] POST /identity/v3/users => generated 287 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 20:01:08.652612 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e144a5a1-bcbc-4642-a5d1-06c46a87546a admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:01:08.663638 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e144a5a1-bcbc-4642-a5d1-06c46a87546a admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:01:08.668295 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f993fd85-a5fb-4d32-a298-9770138c4480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:01:08.668891 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f993fd85-a5fb-4d32-a298-9770138c4480 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:01:08.669165 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f993fd85-a5fb-4d32-a298-9770138c4480 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:01:08.669433 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f993fd85-a5fb-4d32-a298-9770138c4480 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:01:08.670072 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f993fd85-a5fb-4d32-a298-9770138c4480 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:01:08.671739 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f993fd85-a5fb-4d32-a298-9770138c4480 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:01:08.681821 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1369/2742] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 20:01:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 20:01:08.685787 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1211a92c-157e-421a-a5f8-5b5f422ff9d0 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:01:08.696165 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1211a92c-157e-421a-a5f8-5b5f422ff9d0 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:01:08.701415 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e221ee83-f390-4c18-a4ad-ba5e531963c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:01:08.702021 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e221ee83-f390-4c18-a4ad-ba5e531963c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:01:08.702295 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e221ee83-f390-4c18-a4ad-ba5e531963c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:01:08.702570 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e221ee83-f390-4c18-a4ad-ba5e531963c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:01:08.703217 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e221ee83-f390-4c18-a4ad-ba5e531963c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:01:08.705146 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e221ee83-f390-4c18-a4ad-ba5e531963c1 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:01:08.710868 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:01:08.719143 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1374/2743] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 20:01:08 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 20:01:08.722817 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f993fd85-a5fb-4d32-a298-9770138c4480 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:01:08.733377 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f993fd85-a5fb-4d32-a298-9770138c4480 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:01:08.738008 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188ed769-24d6-44bf-89f2-1425f9fb8093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:01:08.738667 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188ed769-24d6-44bf-89f2-1425f9fb8093 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:01:08.738939 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188ed769-24d6-44bf-89f2-1425f9fb8093 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:01:08.739205 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188ed769-24d6-44bf-89f2-1425f9fb8093 admin admin] PATH_INFO: `/v3/projects/e769d7a29dbc470f9013344d9c817d63/users/ae3afb9ab5c3485287cc352735a159b8/roles/74a40bc819cb42eea7e966a4b2a016f1` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:01:08.754723 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188ed769-24d6-44bf-89f2-1425f9fb8093 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e769d7a29dbc470f9013344d9c817d63, user_id=ae3afb9ab5c3485287cc352735a159b8, role_id=74a40bc819cb42eea7e966a4b2a016f1)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:01:08.757431 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188ed769-24d6-44bf-89f2-1425f9fb8093 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:01:08.771825 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1370/2744] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 20:01:08 2022] PUT /identity/v3/projects/e769d7a29dbc470f9013344d9c817d63/users/ae3afb9ab5c3485287cc352735a159b8/roles/74a40bc819cb42eea7e966a4b2a016f1 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 20:01:08.777271 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7c40ad-2bfb-448f-b41a-4e876f8857c2 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:01:08.777797 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7c40ad-2bfb-448f-b41a-4e876f8857c2 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:01:08.778194 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7c40ad-2bfb-448f-b41a-4e876f8857c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:01:08.803485 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-ed7c40ad-2bfb-448f-b41a-4e876f8857c2 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 20:01:08.804676 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-ed7c40ad-2bfb-448f-b41a-4e876f8857c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:01:08.866894 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1375/2745] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 20:01:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 20:01:08.876884 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188ed769-24d6-44bf-89f2-1425f9fb8093 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:01:08.878616 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-188ed769-24d6-44bf-89f2-1425f9fb8093 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:01:08.945616 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188ed769-24d6-44bf-89f2-1425f9fb8093 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:01:08.949406 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b6f027-09be-4140-88be-f561e313c2fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:01:08.950022 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b6f027-09be-4140-88be-f561e313c2fe service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:01:08.950309 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b6f027-09be-4140-88be-f561e313c2fe service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:01:08.951092 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b6f027-09be-4140-88be-f561e313c2fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:01:08.962155 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b6f027-09be-4140-88be-f561e313c2fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:01:08.963585 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b6f027-09be-4140-88be-f561e313c2fe service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:01:08.974629 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1371/2746] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 20:01:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 20:01:13.142248 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d973e1-171e-4c26-a49c-c82eb887e2db None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:01:13.142677 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d973e1-171e-4c26-a49c-c82eb887e2db None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:01:13.142933 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d973e1-171e-4c26-a49c-c82eb887e2db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:01:13.174337 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-34d973e1-171e-4c26-a49c-c82eb887e2db None None] MFA Rules not processed for user `ae3afb9ab5c3485287cc352735a159b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 20:01:13.175520 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-34d973e1-171e-4c26-a49c-c82eb887e2db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:01:13.254264 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1376/2747] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 20:01:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 20:01:13.260524 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b6f027-09be-4140-88be-f561e313c2fe service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:01:13.271199 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b6f027-09be-4140-88be-f561e313c2fe service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:01:13.276263 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef2ba4c-fbbb-47e4-beae-1f17fceb1d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:01:13.276882 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef2ba4c-fbbb-47e4-beae-1f17fceb1d8b admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:01:13.277163 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef2ba4c-fbbb-47e4-beae-1f17fceb1d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:01:13.277442 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef2ba4c-fbbb-47e4-beae-1f17fceb1d8b admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:01:13.278064 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef2ba4c-fbbb-47e4-beae-1f17fceb1d8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:01:13.280843 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef2ba4c-fbbb-47e4-beae-1f17fceb1d8b admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:01:13.315844 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1372/2748] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 20:01:13 2022] POST /identity/v3/projects => generated 371 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 20:01:13.320293 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d973e1-171e-4c26-a49c-c82eb887e2db None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:01:13.332824 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d973e1-171e-4c26-a49c-c82eb887e2db None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:01:13.438987 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59eb80b5-94f0-4b06-9ebc-cb995aebf235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:01:13.439659 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eb80b5-94f0-4b06-9ebc-cb995aebf235 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:01:13.439955 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eb80b5-94f0-4b06-9ebc-cb995aebf235 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:01:13.440228 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eb80b5-94f0-4b06-9ebc-cb995aebf235 admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:01:13.440876 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59eb80b5-94f0-4b06-9ebc-cb995aebf235 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:01:13.442817 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59eb80b5-94f0-4b06-9ebc-cb995aebf235 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:01:13.475516 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1377/2749] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 20:01:13 2022] POST /identity/v3/users => generated 292 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 20:01:13.477759 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef2ba4c-fbbb-47e4-beae-1f17fceb1d8b admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:01:13.489243 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef2ba4c-fbbb-47e4-beae-1f17fceb1d8b admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:01:13.494517 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6dfc2c-f672-449d-9858-339557947373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:01:13.495230 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6dfc2c-f672-449d-9858-339557947373 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:01:13.495397 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6dfc2c-f672-449d-9858-339557947373 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:01:13.495748 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6dfc2c-f672-449d-9858-339557947373 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:01:13.496440 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6dfc2c-f672-449d-9858-339557947373 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:01:13.498567 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6dfc2c-f672-449d-9858-339557947373 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:01:13.510012 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1373/2750] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 20:01:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 20:01:13.515003 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59eb80b5-94f0-4b06-9ebc-cb995aebf235 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:01:13.526250 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59eb80b5-94f0-4b06-9ebc-cb995aebf235 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:01:13.531380 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718a59d5-9e97-4207-a27a-ebf01d4302e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:01:13.532105 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718a59d5-9e97-4207-a27a-ebf01d4302e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:01:13.532384 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718a59d5-9e97-4207-a27a-ebf01d4302e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:01:13.532651 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718a59d5-9e97-4207-a27a-ebf01d4302e2 admin admin] PATH_INFO: `/v3/projects/299cd3a442a34d1fa79f4ccce35b3fb4/users/6ac28b13543d4dc98178965372ef765d/roles/163c30c3b3c246beb849ce176f02d9da` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:01:13.549564 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718a59d5-9e97-4207-a27a-ebf01d4302e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=299cd3a442a34d1fa79f4ccce35b3fb4, user_id=6ac28b13543d4dc98178965372ef765d, role_id=163c30c3b3c246beb849ce176f02d9da)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:01:13.554838 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718a59d5-9e97-4207-a27a-ebf01d4302e2 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:01:13.557082 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:01:13.583775 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1378/2751] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 20:01:13 2022] PUT /identity/v3/projects/299cd3a442a34d1fa79f4ccce35b3fb4/users/6ac28b13543d4dc98178965372ef765d/roles/163c30c3b3c246beb849ce176f02d9da => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 20:01:13.587938 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6dfc2c-f672-449d-9858-339557947373 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:01:13.602065 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6dfc2c-f672-449d-9858-339557947373 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:01:13.668659 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af8d2c7-1cfb-4b3d-8515-c23560ce0aad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:01:13.669320 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af8d2c7-1cfb-4b3d-8515-c23560ce0aad admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:01:13.669589 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af8d2c7-1cfb-4b3d-8515-c23560ce0aad admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:01:13.669857 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af8d2c7-1cfb-4b3d-8515-c23560ce0aad admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:01:13.670513 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af8d2c7-1cfb-4b3d-8515-c23560ce0aad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:01:13.672171 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af8d2c7-1cfb-4b3d-8515-c23560ce0aad admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:01:13.678023 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:01:13.689459 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1374/2752] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 20:01:13 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 20:01:13.690838 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718a59d5-9e97-4207-a27a-ebf01d4302e2 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:01:13.703183 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718a59d5-9e97-4207-a27a-ebf01d4302e2 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:01:13.708580 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3197197e-4d70-47b9-a6ff-900b88839e32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:01:13.709264 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3197197e-4d70-47b9-a6ff-900b88839e32 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:01:13.709545 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3197197e-4d70-47b9-a6ff-900b88839e32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:01:13.709824 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3197197e-4d70-47b9-a6ff-900b88839e32 admin admin] PATH_INFO: `/v3/projects/299cd3a442a34d1fa79f4ccce35b3fb4/users/6ac28b13543d4dc98178965372ef765d/roles/163c30c3b3c246beb849ce176f02d9da` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:01:13.712007 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3197197e-4d70-47b9-a6ff-900b88839e32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=299cd3a442a34d1fa79f4ccce35b3fb4, user_id=6ac28b13543d4dc98178965372ef765d, role_id=163c30c3b3c246beb849ce176f02d9da)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:01:13.714696 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3197197e-4d70-47b9-a6ff-900b88839e32 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:01:13.726534 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1379/2753] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 20:01:13 2022] PUT /identity/v3/projects/299cd3a442a34d1fa79f4ccce35b3fb4/users/6ac28b13543d4dc98178965372ef765d/roles/163c30c3b3c246beb849ce176f02d9da => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 20:01:18.427334 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb23594b-3a0c-4381-a011-9f93124adb62 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:01:18.427957 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb23594b-3a0c-4381-a011-9f93124adb62 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:01:18.427957 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb23594b-3a0c-4381-a011-9f93124adb62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:01:18.462670 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-cb23594b-3a0c-4381-a011-9f93124adb62 None None] MFA Rules not processed for user `6ac28b13543d4dc98178965372ef765d`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 20:01:18.462670 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-cb23594b-3a0c-4381-a011-9f93124adb62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:01:18.548732 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1375/2754] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 20:01:18 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 20:01:18.571530 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3197197e-4d70-47b9-a6ff-900b88839e32 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:01:18.573768 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-3197197e-4d70-47b9-a6ff-900b88839e32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:01:18.686384 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3197197e-4d70-47b9-a6ff-900b88839e32 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:01:18.692557 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7140e843-4f1c-49c4-ab3c-cfe6d082decd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f43e2756b746a6825ca1a06b7d7c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:01:18.693184 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7140e843-4f1c-49c4-ab3c-cfe6d082decd service nova] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:01:18.694060 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7140e843-4f1c-49c4-ab3c-cfe6d082decd service nova] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:01:18.694060 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7140e843-4f1c-49c4-ab3c-cfe6d082decd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:01:18.777759 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7140e843-4f1c-49c4-ab3c-cfe6d082decd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:01:18.780964 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7140e843-4f1c-49c4-ab3c-cfe6d082decd service nova] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:01:18.798316 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1380/2755] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 20:01:18 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 20:01:19.931567 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b0d6df-5249-4797-809a-55c21d4884f7 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:01:19.932158 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b0d6df-5249-4797-809a-55c21d4884f7 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:01:19.932158 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b0d6df-5249-4797-809a-55c21d4884f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:01:19.960157 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-52b0d6df-5249-4797-809a-55c21d4884f7 None None] MFA Rules not processed for user `ae3afb9ab5c3485287cc352735a159b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 20:01:19.964726 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-52b0d6df-5249-4797-809a-55c21d4884f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:01:19.972407 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1376/2756] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 20:01:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 20:01:28.789289 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=63,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:01:29.956990 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=37 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:01:44.420500 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7140e843-4f1c-49c4-ab3c-cfe6d082decd service nova] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:01:44.477029 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7140e843-4f1c-49c4-ab3c-cfe6d082decd service nova] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:01:44.480679 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73be5ed-6f90-454c-b78a-287e86416ed8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:01:44.481287 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73be5ed-6f90-454c-b78a-287e86416ed8 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:01:44.481561 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73be5ed-6f90-454c-b78a-287e86416ed8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:01:44.481824 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73be5ed-6f90-454c-b78a-287e86416ed8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:01:44.483961 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-a73be5ed-6f90-454c-b78a-287e86416ed8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:01:44.498441 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73be5ed-6f90-454c-b78a-287e86416ed8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:01:44.500431 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73be5ed-6f90-454c-b78a-287e86416ed8 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:01:44.512033 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1381/2757] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 20:01:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 20:01:45.705602 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b0d6df-5249-4797-809a-55c21d4884f7 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:01:45.707215 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-52b0d6df-5249-4797-809a-55c21d4884f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:01:45.720325 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b0d6df-5249-4797-809a-55c21d4884f7 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:01:45.724428 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce186c8-f47d-4e51-b175-223b3841a94f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:01:45.725221 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce186c8-f47d-4e51-b175-223b3841a94f service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:01:45.725621 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce186c8-f47d-4e51-b175-223b3841a94f service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:01:45.725892 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce186c8-f47d-4e51-b175-223b3841a94f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:01:45.727741 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-2ce186c8-f47d-4e51-b175-223b3841a94f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:01:45.742692 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce186c8-f47d-4e51-b175-223b3841a94f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:01:45.744186 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce186c8-f47d-4e51-b175-223b3841a94f service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:01:45.757052 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1377/2758] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 20:01:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 20:01:46.420627 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73be5ed-6f90-454c-b78a-287e86416ed8 service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:01:46.432015 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73be5ed-6f90-454c-b78a-287e86416ed8 service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:01:46.437929 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df696a54-cb8a-4348-a900-74e93b7b564a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:01:46.437929 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df696a54-cb8a-4348-a900-74e93b7b564a service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:01:46.437929 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df696a54-cb8a-4348-a900-74e93b7b564a service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:01:46.444161 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df696a54-cb8a-4348-a900-74e93b7b564a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:01:46.447300 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-df696a54-cb8a-4348-a900-74e93b7b564a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:01:46.465522 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df696a54-cb8a-4348-a900-74e93b7b564a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:01:46.467708 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df696a54-cb8a-4348-a900-74e93b7b564a service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:01:46.482106 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1382/2759] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 20:01:46 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 20:01:46.503211 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce186c8-f47d-4e51-b175-223b3841a94f service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:01:46.507509 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-2ce186c8-f47d-4e51-b175-223b3841a94f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:01:46.539559 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce186c8-f47d-4e51-b175-223b3841a94f service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:01:46.551740 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21b17b5-e846-4972-bd97-0b18337b3f85 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f43e2756b746a6825ca1a06b7d7c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:01:46.552894 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21b17b5-e846-4972-bd97-0b18337b3f85 service nova] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:01:46.553429 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21b17b5-e846-4972-bd97-0b18337b3f85 service nova] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:01:46.554002 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21b17b5-e846-4972-bd97-0b18337b3f85 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:01:46.558481 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-a21b17b5-e846-4972-bd97-0b18337b3f85 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:01:46.581651 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21b17b5-e846-4972-bd97-0b18337b3f85 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:01:46.584821 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21b17b5-e846-4972-bd97-0b18337b3f85 service nova] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:01:46.608634 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1378/2760] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 20:01:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 20:01:48.089178 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df696a54-cb8a-4348-a900-74e93b7b564a service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:01:48.090799 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-df696a54-cb8a-4348-a900-74e93b7b564a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:01:48.197198 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df696a54-cb8a-4348-a900-74e93b7b564a service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:01:48.203599 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01423847-2b8c-4051-ab41-993ccfb5d1d5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eef0f73297e04d62bc22acb790fad6ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:01:48.204500 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01423847-2b8c-4051-ab41-993ccfb5d1d5 service placement] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:01:48.204773 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01423847-2b8c-4051-ab41-993ccfb5d1d5 service placement] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:01:48.205039 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01423847-2b8c-4051-ab41-993ccfb5d1d5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:01:48.206831 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-01423847-2b8c-4051-ab41-993ccfb5d1d5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:01:48.219501 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01423847-2b8c-4051-ab41-993ccfb5d1d5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:01:48.221154 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01423847-2b8c-4051-ab41-993ccfb5d1d5 service placement] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:01:48.251283 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1383/2761] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 20:01:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 20:01:50.793068 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21b17b5-e846-4972-bd97-0b18337b3f85 service nova] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:01:50.807158 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21b17b5-e846-4972-bd97-0b18337b3f85 service nova] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:01:50.813631 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed7945e-e465-4863-a67a-f39816fe6b26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:01:50.814681 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed7945e-e465-4863-a67a-f39816fe6b26 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:01:50.815501 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed7945e-e465-4863-a67a-f39816fe6b26 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:01:50.815501 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed7945e-e465-4863-a67a-f39816fe6b26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:01:50.817631 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-3ed7945e-e465-4863-a67a-f39816fe6b26 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:01:50.902810 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed7945e-e465-4863-a67a-f39816fe6b26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:01:50.904400 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed7945e-e465-4863-a67a-f39816fe6b26 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:01:50.927853 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1379/2762] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 20:01:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 20:01:55.354908 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01423847-2b8c-4051-ab41-993ccfb5d1d5 service placement] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:01:55.357070 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-01423847-2b8c-4051-ab41-993ccfb5d1d5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:01:55.376150 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01423847-2b8c-4051-ab41-993ccfb5d1d5 service placement] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:01:55.381067 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211d04b7-bfbc-4b6d-9da9-7f3d5fd6f195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:01:55.381746 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211d04b7-bfbc-4b6d-9da9-7f3d5fd6f195 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:01:55.382031 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211d04b7-bfbc-4b6d-9da9-7f3d5fd6f195 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:01:55.382333 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211d04b7-bfbc-4b6d-9da9-7f3d5fd6f195 admin admin] PATH_INFO: `/v3/users/8030a8ebdd8c45c7bd30403c866a9278` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:01:55.383566 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211d04b7-bfbc-4b6d-9da9-7f3d5fd6f195 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8030a8ebdd8c45c7bd30403c866a9278)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:01:55.385574 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211d04b7-bfbc-4b6d-9da9-7f3d5fd6f195 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:01:55.442177 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-211d04b7-bfbc-4b6d-9da9-7f3d5fd6f195 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8030a8ebdd8c45c7bd30403c866a9278'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:01:55.442841 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-211d04b7-bfbc-4b6d-9da9-7f3d5fd6f195 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8030a8ebdd8c45c7bd30403c866a9278'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:01:55.450007 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-211d04b7-bfbc-4b6d-9da9-7f3d5fd6f195 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8030a8ebdd8c45c7bd30403c866a9278'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:01:55.458628 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-211d04b7-bfbc-4b6d-9da9-7f3d5fd6f195 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8030a8ebdd8c45c7bd30403c866a9278'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:01:55.466657 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:01:55.472643 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-211d04b7-bfbc-4b6d-9da9-7f3d5fd6f195 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8030a8ebdd8c45c7bd30403c866a9278'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:01:55.485675 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1384/2763] 10.222.0.61 () {58 vars in 1224 bytes} [Mon Aug 22 20:01:55 2022] DELETE /identity/v3/users/8030a8ebdd8c45c7bd30403c866a9278 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 20:01:55.715422 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed7945e-e465-4863-a67a-f39816fe6b26 service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:01:55.728424 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed7945e-e465-4863-a67a-f39816fe6b26 service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:01:55.810110 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b2c789-0287-49ce-96b5-4a6493bd5ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:01:55.810931 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b2c789-0287-49ce-96b5-4a6493bd5ba2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:01:55.811279 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b2c789-0287-49ce-96b5-4a6493bd5ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:01:55.811722 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b2c789-0287-49ce-96b5-4a6493bd5ba2 admin admin] PATH_INFO: `/v3/projects/0e3308fc667c4b8c8552e27ba69d40fc` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:01:55.813049 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b2c789-0287-49ce-96b5-4a6493bd5ba2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e3308fc667c4b8c8552e27ba69d40fc)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:01:55.815042 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b2c789-0287-49ce-96b5-4a6493bd5ba2 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:01:55.872116 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-68b2c789-0287-49ce-96b5-4a6493bd5ba2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e3308fc667c4b8c8552e27ba69d40fc', 'request_id': 'req-68b2c789-0287-49ce-96b5-4a6493bd5ba2'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:01:55.880631 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-68b2c789-0287-49ce-96b5-4a6493bd5ba2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e3308fc667c4b8c8552e27ba69d40fc', 'request_id': 'req-68b2c789-0287-49ce-96b5-4a6493bd5ba2'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:01:55.888210 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-68b2c789-0287-49ce-96b5-4a6493bd5ba2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e3308fc667c4b8c8552e27ba69d40fc', 'request_id': 'req-68b2c789-0287-49ce-96b5-4a6493bd5ba2'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:01:55.896066 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:01:55.902443 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-68b2c789-0287-49ce-96b5-4a6493bd5ba2 admin admin] The token cache is being invalidate because project 0e3308fc667c4b8c8552e27ba69d40fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 20:01:55.902741 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-68b2c789-0287-49ce-96b5-4a6493bd5ba2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:01:55.903607 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-68b2c789-0287-49ce-96b5-4a6493bd5ba2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:01:55.911089 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1380/2764] 10.222.0.61 () {58 vars in 1233 bytes} [Mon Aug 22 20:01:55 2022] DELETE /identity/v3/projects/0e3308fc667c4b8c8552e27ba69d40fc => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 20:01:55.912574 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a33c5e-13c5-44a5-bd5b-433453a74ac2 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:01:55.912866 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a33c5e-13c5-44a5-bd5b-433453a74ac2 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:01:55.913142 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a33c5e-13c5-44a5-bd5b-433453a74ac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:01:55.933248 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-07a33c5e-13c5-44a5-bd5b-433453a74ac2 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 20:01:55.934474 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-07a33c5e-13c5-44a5-bd5b-433453a74ac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:01:56.001088 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1385/2765] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 20:01:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 20:01:56.009400 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7608b47e-3c31-45a7-bac3-72344b1485e3 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:01:56.009683 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7608b47e-3c31-45a7-bac3-72344b1485e3 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:01:56.009964 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7608b47e-3c31-45a7-bac3-72344b1485e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:01:56.031200 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-7608b47e-3c31-45a7-bac3-72344b1485e3 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 20:01:56.032472 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-7608b47e-3c31-45a7-bac3-72344b1485e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:01:56.047628 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1381/2766] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 20:01:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 20:01:56.052195 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a33c5e-13c5-44a5-bd5b-433453a74ac2 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:01:56.062464 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a33c5e-13c5-44a5-bd5b-433453a74ac2 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:01:56.067513 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa34b11-5915-4a3a-a6ae-9dac5013e3b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:01:56.068186 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa34b11-5915-4a3a-a6ae-9dac5013e3b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:01:56.068466 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa34b11-5915-4a3a-a6ae-9dac5013e3b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:01:56.068741 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa34b11-5915-4a3a-a6ae-9dac5013e3b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:01:56.069413 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa34b11-5915-4a3a-a6ae-9dac5013e3b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 20:01:56.069897 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa34b11-5915-4a3a-a6ae-9dac5013e3b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:01:56.071610 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa34b11-5915-4a3a-a6ae-9dac5013e3b0 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:01:56.098863 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1386/2767] 10.222.0.61 () {58 vars in 1153 bytes} [Mon Aug 22 20:01:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 20:01:56.102711 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7608b47e-3c31-45a7-bac3-72344b1485e3 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:01:56.114915 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7608b47e-3c31-45a7-bac3-72344b1485e3 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:01:56.120000 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44148138-e111-498f-9543-4d1469b30395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:01:56.120619 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44148138-e111-498f-9543-4d1469b30395 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:01:56.120905 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44148138-e111-498f-9543-4d1469b30395 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:01:56.121151 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44148138-e111-498f-9543-4d1469b30395 admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:01:56.121786 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44148138-e111-498f-9543-4d1469b30395 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:01:56.123691 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44148138-e111-498f-9543-4d1469b30395 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:01:56.155501 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1382/2768] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 20:01:56 2022] POST /identity/v3/projects => generated 387 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 20:01:56.160175 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa34b11-5915-4a3a-a6ae-9dac5013e3b0 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:01:56.170716 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa34b11-5915-4a3a-a6ae-9dac5013e3b0 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:01:56.233425 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da98059-acd2-4996-9b26-2de15cae4ec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:01:56.234046 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da98059-acd2-4996-9b26-2de15cae4ec5 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:01:56.234326 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da98059-acd2-4996-9b26-2de15cae4ec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:01:56.234667 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da98059-acd2-4996-9b26-2de15cae4ec5 admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:01:56.235250 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da98059-acd2-4996-9b26-2de15cae4ec5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:01:56.237231 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da98059-acd2-4996-9b26-2de15cae4ec5 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:01:56.260508 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1387/2769] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 20:01:56 2022] POST /identity/v3/users => generated 294 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 20:01:56.265364 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44148138-e111-498f-9543-4d1469b30395 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:01:56.276311 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44148138-e111-498f-9543-4d1469b30395 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:01:56.281825 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f119a895-57bd-454d-a65f-8d21cc793a9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:01:56.282444 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f119a895-57bd-454d-a65f-8d21cc793a9b admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:01:56.282725 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f119a895-57bd-454d-a65f-8d21cc793a9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:01:56.283000 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f119a895-57bd-454d-a65f-8d21cc793a9b admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:01:56.283669 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f119a895-57bd-454d-a65f-8d21cc793a9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:01:56.285320 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f119a895-57bd-454d-a65f-8d21cc793a9b admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:01:56.297012 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1383/2770] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 20:01:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 20:01:56.301191 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da98059-acd2-4996-9b26-2de15cae4ec5 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:01:56.325753 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da98059-acd2-4996-9b26-2de15cae4ec5 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:01:56.337173 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4809ae26-29c6-41ce-b08a-0ada9809c5c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:01:56.338029 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4809ae26-29c6-41ce-b08a-0ada9809c5c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:01:56.338602 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4809ae26-29c6-41ce-b08a-0ada9809c5c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:01:56.339061 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4809ae26-29c6-41ce-b08a-0ada9809c5c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:01:56.339928 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4809ae26-29c6-41ce-b08a-0ada9809c5c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:01:56.341868 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4809ae26-29c6-41ce-b08a-0ada9809c5c4 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:01:56.354933 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1388/2771] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 20:01:56 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 20:01:56.359039 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f119a895-57bd-454d-a65f-8d21cc793a9b admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:01:56.369708 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f119a895-57bd-454d-a65f-8d21cc793a9b admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:01:56.377624 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfc2ff5-97c3-4558-8bef-da1c508b7e1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:01:56.378305 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfc2ff5-97c3-4558-8bef-da1c508b7e1d admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:01:56.378577 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfc2ff5-97c3-4558-8bef-da1c508b7e1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:01:56.379161 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfc2ff5-97c3-4558-8bef-da1c508b7e1d admin admin] PATH_INFO: `/v3/projects/548a965fed81436fa0440d4c4db00e2c/users/e0dc5256349a442db02b6e5584872356/roles/74a40bc819cb42eea7e966a4b2a016f1` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:01:56.401560 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfc2ff5-97c3-4558-8bef-da1c508b7e1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=548a965fed81436fa0440d4c4db00e2c, user_id=e0dc5256349a442db02b6e5584872356, role_id=74a40bc819cb42eea7e966a4b2a016f1)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:01:56.404671 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfc2ff5-97c3-4558-8bef-da1c508b7e1d admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:01:56.423108 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1384/2772] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 20:01:56 2022] PUT /identity/v3/projects/548a965fed81436fa0440d4c4db00e2c/users/e0dc5256349a442db02b6e5584872356/roles/74a40bc819cb42eea7e966a4b2a016f1 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 20:01:56.429810 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910920b0-d508-4710-ac77-af27092c2fe4 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:01:56.430327 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910920b0-d508-4710-ac77-af27092c2fe4 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:01:56.430769 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910920b0-d508-4710-ac77-af27092c2fe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:01:56.457502 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-910920b0-d508-4710-ac77-af27092c2fe4 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 20:01:56.458767 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-910920b0-d508-4710-ac77-af27092c2fe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:01:56.532050 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:01:56.544109 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1389/2773] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 20:01:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 20:01:56.554618 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfc2ff5-97c3-4558-8bef-da1c508b7e1d admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:01:56.556502 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-3dfc2ff5-97c3-4558-8bef-da1c508b7e1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:01:56.614755 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfc2ff5-97c3-4558-8bef-da1c508b7e1d admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:01:56.618426 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84485cd7-c620-4368-91ba-67cc933ba22b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:01:56.619033 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84485cd7-c620-4368-91ba-67cc933ba22b service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:01:56.619303 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84485cd7-c620-4368-91ba-67cc933ba22b service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:01:56.619573 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84485cd7-c620-4368-91ba-67cc933ba22b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:01:56.629881 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84485cd7-c620-4368-91ba-67cc933ba22b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:01:56.631616 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84485cd7-c620-4368-91ba-67cc933ba22b service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:01:56.647662 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1385/2774] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 20:01:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 20:02:01.141042 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7bbf0e-097d-4469-9207-d218da005a45 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:02:01.141606 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7bbf0e-097d-4469-9207-d218da005a45 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:02:01.141606 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7bbf0e-097d-4469-9207-d218da005a45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:02:01.173286 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-eb7bbf0e-097d-4469-9207-d218da005a45 None None] MFA Rules not processed for user `e0dc5256349a442db02b6e5584872356`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 20:02:01.174566 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-eb7bbf0e-097d-4469-9207-d218da005a45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:02:01.403040 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1390/2775] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 20:02:01 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 20:02:01.411768 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84485cd7-c620-4368-91ba-67cc933ba22b service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:02:01.414035 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-84485cd7-c620-4368-91ba-67cc933ba22b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:02:01.514709 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84485cd7-c620-4368-91ba-67cc933ba22b service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:02:01.521108 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271f4b06-a2b9-43a3-8c86-07530921234e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f43e2756b746a6825ca1a06b7d7c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:02:01.521729 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271f4b06-a2b9-43a3-8c86-07530921234e service nova] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:02:01.522010 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271f4b06-a2b9-43a3-8c86-07530921234e service nova] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:02:01.522282 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271f4b06-a2b9-43a3-8c86-07530921234e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:02:01.532094 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271f4b06-a2b9-43a3-8c86-07530921234e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:02:01.533693 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271f4b06-a2b9-43a3-8c86-07530921234e service nova] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:02:01.543378 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1386/2776] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 20:02:01 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 20:02:03.052311 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d839744-6d3e-4bcc-8044-45a77d84a8f9 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:02:03.053125 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d839744-6d3e-4bcc-8044-45a77d84a8f9 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:02:03.053125 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d839744-6d3e-4bcc-8044-45a77d84a8f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:02:03.072460 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-6d839744-6d3e-4bcc-8044-45a77d84a8f9 None None] MFA Rules not processed for user `e0dc5256349a442db02b6e5584872356`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 20:02:03.073712 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-6d839744-6d3e-4bcc-8044-45a77d84a8f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:02:03.082063 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1391/2777] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 20:02:03 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 20:02:03.442007 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271f4b06-a2b9-43a3-8c86-07530921234e service nova] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:02:03.454741 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271f4b06-a2b9-43a3-8c86-07530921234e service nova] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:02:03.460790 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7274281-bf3b-4111-ae58-e7cfe655f9f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:02:03.461768 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7274281-bf3b-4111-ae58-e7cfe655f9f0 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:02:03.462118 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7274281-bf3b-4111-ae58-e7cfe655f9f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:02:03.462564 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7274281-bf3b-4111-ae58-e7cfe655f9f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:02:03.464720 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-a7274281-bf3b-4111-ae58-e7cfe655f9f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:02:03.472351 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:02:03.477090 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7274281-bf3b-4111-ae58-e7cfe655f9f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:02:03.479595 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7274281-bf3b-4111-ae58-e7cfe655f9f0 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:02:03.491072 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1387/2778] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 20:02:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 20:02:05.744044 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d839744-6d3e-4bcc-8044-45a77d84a8f9 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:02:05.760622 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d839744-6d3e-4bcc-8044-45a77d84a8f9 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:02:05.767990 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc11606-0c8b-4a7e-8a5f-646096fd4513 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f43e2756b746a6825ca1a06b7d7c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:02:05.769025 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc11606-0c8b-4a7e-8a5f-646096fd4513 service nova] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:02:05.769364 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc11606-0c8b-4a7e-8a5f-646096fd4513 service nova] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:02:05.769702 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc11606-0c8b-4a7e-8a5f-646096fd4513 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:02:05.771842 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-2bc11606-0c8b-4a7e-8a5f-646096fd4513 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:02:05.788921 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc11606-0c8b-4a7e-8a5f-646096fd4513 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:02:05.791348 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc11606-0c8b-4a7e-8a5f-646096fd4513 service nova] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:02:05.804508 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1392/2779] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 20:02:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 20:02:09.024947 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7274281-bf3b-4111-ae58-e7cfe655f9f0 service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:02:09.033957 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7274281-bf3b-4111-ae58-e7cfe655f9f0 service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:02:09.038514 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70d9f63-e09b-4a18-8fa2-0aad2ae45edd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:02:09.039329 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70d9f63-e09b-4a18-8fa2-0aad2ae45edd service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:02:09.039618 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70d9f63-e09b-4a18-8fa2-0aad2ae45edd service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:02:09.039933 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70d9f63-e09b-4a18-8fa2-0aad2ae45edd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:02:09.042100 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-f70d9f63-e09b-4a18-8fa2-0aad2ae45edd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:02:09.109133 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70d9f63-e09b-4a18-8fa2-0aad2ae45edd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:02:09.110598 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70d9f63-e09b-4a18-8fa2-0aad2ae45edd service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:02:09.122230 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1388/2780] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 20:02:09 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 20:02:10.971479 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc11606-0c8b-4a7e-8a5f-646096fd4513 service nova] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:02:10.973098 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-2bc11606-0c8b-4a7e-8a5f-646096fd4513 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:02:10.988040 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc11606-0c8b-4a7e-8a5f-646096fd4513 service nova] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:02:10.993661 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a65443-c5ec-486e-a773-888eb6f9ba26 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f43e2756b746a6825ca1a06b7d7c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:02:10.994357 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a65443-c5ec-486e-a773-888eb6f9ba26 service nova] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:02:10.994702 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a65443-c5ec-486e-a773-888eb6f9ba26 service nova] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:02:10.995156 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a65443-c5ec-486e-a773-888eb6f9ba26 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:02:10.997070 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-62a65443-c5ec-486e-a773-888eb6f9ba26 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:02:11.011093 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a65443-c5ec-486e-a773-888eb6f9ba26 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:02:11.013097 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a65443-c5ec-486e-a773-888eb6f9ba26 service nova] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:02:11.026731 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1393/2781] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 20:02:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 20:02:19.116983 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=33 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:02:21.022226 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=48 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:03:01.487797 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70d9f63-e09b-4a18-8fa2-0aad2ae45edd service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:03:01.489212 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-f70d9f63-e09b-4a18-8fa2-0aad2ae45edd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:03:01.561743 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70d9f63-e09b-4a18-8fa2-0aad2ae45edd service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:03:01.566712 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedca3b3-ce7a-49de-8e61-ab7238dd7189 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eef0f73297e04d62bc22acb790fad6ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:03:01.567315 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedca3b3-ce7a-49de-8e61-ab7238dd7189 service placement] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:03:01.567590 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedca3b3-ce7a-49de-8e61-ab7238dd7189 service placement] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:03:01.567889 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedca3b3-ce7a-49de-8e61-ab7238dd7189 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:03:01.569634 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-cedca3b3-ce7a-49de-8e61-ab7238dd7189 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:03:01.582159 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedca3b3-ce7a-49de-8e61-ab7238dd7189 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:03:01.583760 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedca3b3-ce7a-49de-8e61-ab7238dd7189 service placement] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:03:01.594217 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1389/2782] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 20:03:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 20:03:11.589727 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=32 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:03:54.116242 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e331aa1-ca1f-4199-b5a3-b15e003bd62c None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:03:54.119672 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e331aa1-ca1f-4199-b5a3-b15e003bd62c None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:03:54.119672 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e331aa1-ca1f-4199-b5a3-b15e003bd62c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:03:54.179183 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-5e331aa1-ca1f-4199-b5a3-b15e003bd62c None None] MFA Rules not processed for user `399396a8fe254da199bc48b0ca5243d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 20:03:54.180968 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-5e331aa1-ca1f-4199-b5a3-b15e003bd62c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:03:54.268452 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1394/2783] 10.222.0.61 () {58 vars in 1009 bytes} [Mon Aug 22 20:03:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 20:03:54.321446 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedca3b3-ce7a-49de-8e61-ab7238dd7189 service placement] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:03:54.322874 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-cedca3b3-ce7a-49de-8e61-ab7238dd7189 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:03:54.428683 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedca3b3-ce7a-49de-8e61-ab7238dd7189 service placement] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:03:54.434293 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4767b5dc-aa27-4af1-8032-1736327949a1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2824359491934d0dacaa21a718becee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:03:54.434910 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4767b5dc-aa27-4af1-8032-1736327949a1 service swift] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:03:54.435186 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4767b5dc-aa27-4af1-8032-1736327949a1 service swift] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:03:54.435458 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4767b5dc-aa27-4af1-8032-1736327949a1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:03:54.444617 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4767b5dc-aa27-4af1-8032-1736327949a1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:03:54.447267 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4767b5dc-aa27-4af1-8032-1736327949a1 service swift] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:03:54.458883 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1390/2784] 10.222.0.61 () {60 vars in 1384 bytes} [Mon Aug 22 20:03:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 22 20:04:04.265341 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=40 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:04:04.455182 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=41 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:04:07.796018 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a3308a-89ad-4368-ae28-ec00543c2a19 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:04:07.796806 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a3308a-89ad-4368-ae28-ec00543c2a19 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:04:07.796806 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a3308a-89ad-4368-ae28-ec00543c2a19 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:04:07.798302 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1395/2785] 10.222.0.61 () {54 vars in 921 bytes} [Mon Aug 22 20:04:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 20:04:07.804796 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e6e1db-bf59-475b-8e2f-eacdd4292cab None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:04:07.805349 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e6e1db-bf59-475b-8e2f-eacdd4292cab None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:04:07.805349 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e6e1db-bf59-475b-8e2f-eacdd4292cab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:04:07.846765 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-48e6e1db-bf59-475b-8e2f-eacdd4292cab None None] MFA Rules not processed for user `e7f43e2756b746a6825ca1a06b7d7c91`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 20:04:07.848143 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-48e6e1db-bf59-475b-8e2f-eacdd4292cab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:04:07.858213 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1391/2786] 10.222.0.61 () {58 vars in 1018 bytes} [Mon Aug 22 20:04:07 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 20:04:07.869784 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a3308a-89ad-4368-ae28-ec00543c2a19 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:04:07.872098 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-85a3308a-89ad-4368-ae28-ec00543c2a19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:04:07.956748 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a3308a-89ad-4368-ae28-ec00543c2a19 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:04:07.960332 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbae804f-3386-42b8-a477-251a7ae367e9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d6a79ed44e5396d13cd29ba160ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:04:07.960942 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbae804f-3386-42b8-a477-251a7ae367e9 service glance] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:04:07.961509 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbae804f-3386-42b8-a477-251a7ae367e9 service glance] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:04:07.961509 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbae804f-3386-42b8-a477-251a7ae367e9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:04:07.973001 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbae804f-3386-42b8-a477-251a7ae367e9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:04:07.974491 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbae804f-3386-42b8-a477-251a7ae367e9 service glance] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:04:07.986016 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1396/2787] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 20:04:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 20:04:08.056124 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5112de47-13da-4d7f-8643-a9b53585e770 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:04:08.056581 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5112de47-13da-4d7f-8643-a9b53585e770 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:04:08.056938 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5112de47-13da-4d7f-8643-a9b53585e770 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:04:08.058464 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1392/2788] 10.222.0.61 () {54 vars in 921 bytes} [Mon Aug 22 20:04:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 20:04:08.065475 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a6ccdf-d800-4214-afa2-46ace9ea4516 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:04:08.065759 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a6ccdf-d800-4214-afa2-46ace9ea4516 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:04:08.066058 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a6ccdf-d800-4214-afa2-46ace9ea4516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:04:08.109620 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-e9a6ccdf-d800-4214-afa2-46ace9ea4516 None None] MFA Rules not processed for user `a4aca06c966a42f29f5c5bc108529d41`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 20:04:08.110856 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-e9a6ccdf-d800-4214-afa2-46ace9ea4516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:04:08.117414 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1397/2789] 10.222.0.61 () {58 vars in 1018 bytes} [Mon Aug 22 20:04:08 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 20:04:08.129276 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5112de47-13da-4d7f-8643-a9b53585e770 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:04:08.131073 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-5112de47-13da-4d7f-8643-a9b53585e770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:04:08.144478 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5112de47-13da-4d7f-8643-a9b53585e770 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:04:08.148425 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43146a5-c292-4552-88a8-f5c7390a06db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:04:08.149039 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43146a5-c292-4552-88a8-f5c7390a06db service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:04:08.149368 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43146a5-c292-4552-88a8-f5c7390a06db service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:04:08.149686 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43146a5-c292-4552-88a8-f5c7390a06db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:04:08.157580 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43146a5-c292-4552-88a8-f5c7390a06db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:04:08.159017 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43146a5-c292-4552-88a8-f5c7390a06db service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:04:08.166834 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1393/2790] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 20:04:08 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 20:04:08.252313 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af91b5e-3868-46cf-8a52-79624019dd5b None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:04:08.252623 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af91b5e-3868-46cf-8a52-79624019dd5b None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:04:08.252894 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af91b5e-3868-46cf-8a52-79624019dd5b None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:04:08.254273 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1398/2791] 10.222.0.61 () {54 vars in 921 bytes} [Mon Aug 22 20:04:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 20:04:08.260146 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc3989c-ef0d-494f-96f2-bd8272161161 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:04:08.260505 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc3989c-ef0d-494f-96f2-bd8272161161 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:04:08.260881 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc3989c-ef0d-494f-96f2-bd8272161161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:04:08.281967 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-cfc3989c-ef0d-494f-96f2-bd8272161161 None None] MFA Rules not processed for user `e7f43e2756b746a6825ca1a06b7d7c91`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 20:04:08.283827 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-cfc3989c-ef0d-494f-96f2-bd8272161161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:04:08.295624 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1394/2792] 10.222.0.61 () {58 vars in 1018 bytes} [Mon Aug 22 20:04:08 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 20:04:08.307186 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af91b5e-3868-46cf-8a52-79624019dd5b None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:04:08.321367 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af91b5e-3868-46cf-8a52-79624019dd5b None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:04:08.326412 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42273fc7-4884-4118-bf9d-f669ed48cc3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:04:08.327452 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42273fc7-4884-4118-bf9d-f669ed48cc3b service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:04:08.327766 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42273fc7-4884-4118-bf9d-f669ed48cc3b service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:04:08.328040 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42273fc7-4884-4118-bf9d-f669ed48cc3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:04:08.339084 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42273fc7-4884-4118-bf9d-f669ed48cc3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:04:08.341139 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42273fc7-4884-4118-bf9d-f669ed48cc3b service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:04:08.355034 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1399/2793] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 20:04:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 20:04:10.915950 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8f9884-afcb-466f-b3eb-1127927befc9 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:04:10.916432 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8f9884-afcb-466f-b3eb-1127927befc9 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:04:10.916524 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8f9884-afcb-466f-b3eb-1127927befc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:04:10.935056 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-fb8f9884-afcb-466f-b3eb-1127927befc9 None None] MFA Rules not processed for user `399396a8fe254da199bc48b0ca5243d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 20:04:10.936242 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-fb8f9884-afcb-466f-b3eb-1127927befc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:04:10.943322 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1395/2794] 10.222.0.61 () {58 vars in 1009 bytes} [Mon Aug 22 20:04:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 20:04:10.955835 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42273fc7-4884-4118-bf9d-f669ed48cc3b service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:04:10.967598 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42273fc7-4884-4118-bf9d-f669ed48cc3b service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:04:10.972499 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767e2bbc-98f4-4cfb-ad68-d6c2e1afcc07 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2824359491934d0dacaa21a718becee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:04:10.973085 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e2bbc-98f4-4cfb-ad68-d6c2e1afcc07 service swift] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:04:10.973355 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e2bbc-98f4-4cfb-ad68-d6c2e1afcc07 service swift] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:04:10.973622 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e2bbc-98f4-4cfb-ad68-d6c2e1afcc07 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:04:10.981450 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767e2bbc-98f4-4cfb-ad68-d6c2e1afcc07 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:04:10.983006 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767e2bbc-98f4-4cfb-ad68-d6c2e1afcc07 service swift] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:04:10.990898 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1400/2795] 10.222.0.61 () {60 vars in 1384 bytes} [Mon Aug 22 20:04:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 22 20:04:20.929936 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=21 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:04:20.987717 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=56 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:04:39.074067 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f349e74-a90a-45d9-981b-1d2e6898ef43 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:04:39.074544 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f349e74-a90a-45d9-981b-1d2e6898ef43 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:04:39.074643 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f349e74-a90a-45d9-981b-1d2e6898ef43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:04:39.097659 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-7f349e74-a90a-45d9-981b-1d2e6898ef43 None None] MFA Rules not processed for user `399396a8fe254da199bc48b0ca5243d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 20:04:39.098875 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-7f349e74-a90a-45d9-981b-1d2e6898ef43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:04:39.106567 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1396/2796] 10.222.0.61 () {58 vars in 967 bytes} [Mon Aug 22 20:04:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 20:04:39.119247 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767e2bbc-98f4-4cfb-ad68-d6c2e1afcc07 service swift] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:04:39.132174 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767e2bbc-98f4-4cfb-ad68-d6c2e1afcc07 service swift] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:04:39.137225 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bbd9d9-a825-4269-a89b-644fb2e74b5e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2824359491934d0dacaa21a718becee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:04:39.137821 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bbd9d9-a825-4269-a89b-644fb2e74b5e service swift] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:04:39.138099 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bbd9d9-a825-4269-a89b-644fb2e74b5e service swift] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:04:39.138370 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bbd9d9-a825-4269-a89b-644fb2e74b5e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:04:39.146907 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bbd9d9-a825-4269-a89b-644fb2e74b5e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:04:39.148519 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bbd9d9-a825-4269-a89b-644fb2e74b5e service swift] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:04:39.157282 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1401/2797] 10.222.0.61 () {60 vars in 1384 bytes} [Mon Aug 22 20:04:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 22 20:04:49.092666 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:04:49.153953 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=6 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:04:50.268253 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e19bc41-f2d9-4175-9bcd-2fc2d0ff968c None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:04:50.268936 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e19bc41-f2d9-4175-9bcd-2fc2d0ff968c None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:04:50.268936 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e19bc41-f2d9-4175-9bcd-2fc2d0ff968c None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:04:50.270347 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1397/2798] 10.222.0.61 () {54 vars in 921 bytes} [Mon Aug 22 20:04:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 20:04:50.277035 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5273179a-5874-43c2-ac26-2fe6eea932f7 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:04:50.277626 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5273179a-5874-43c2-ac26-2fe6eea932f7 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:04:50.277626 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5273179a-5874-43c2-ac26-2fe6eea932f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:04:50.298690 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-5273179a-5874-43c2-ac26-2fe6eea932f7 None None] MFA Rules not processed for user `a4aca06c966a42f29f5c5bc108529d41`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 20:04:50.299998 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-5273179a-5874-43c2-ac26-2fe6eea932f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:04:50.308485 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1402/2799] 10.222.0.61 () {58 vars in 1018 bytes} [Mon Aug 22 20:04:50 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 20:04:50.320595 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e19bc41-f2d9-4175-9bcd-2fc2d0ff968c None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:04:50.333082 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e19bc41-f2d9-4175-9bcd-2fc2d0ff968c None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:04:50.336808 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed756ff-ecea-4a18-92c7-779084295336 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:04:50.337410 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed756ff-ecea-4a18-92c7-779084295336 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:04:50.337685 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed756ff-ecea-4a18-92c7-779084295336 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:04:50.337957 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed756ff-ecea-4a18-92c7-779084295336 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:04:50.345759 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed756ff-ecea-4a18-92c7-779084295336 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:04:50.347188 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed756ff-ecea-4a18-92c7-779084295336 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:04:50.355677 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1398/2800] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 20:04:50 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 20:04:51.455628 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e54cae-530d-4514-8590-edd9565fce01 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:04:51.456269 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e54cae-530d-4514-8590-edd9565fce01 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:04:51.456517 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e54cae-530d-4514-8590-edd9565fce01 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:04:51.458098 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1403/2801] 10.222.0.61 () {54 vars in 921 bytes} [Mon Aug 22 20:04:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 20:04:51.465271 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654c0a50-2175-4628-b53d-e2f11c589b3b None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:04:51.465813 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654c0a50-2175-4628-b53d-e2f11c589b3b None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:04:51.466190 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654c0a50-2175-4628-b53d-e2f11c589b3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:04:51.489162 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-654c0a50-2175-4628-b53d-e2f11c589b3b None None] MFA Rules not processed for user `e7f43e2756b746a6825ca1a06b7d7c91`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 20:04:51.490764 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-654c0a50-2175-4628-b53d-e2f11c589b3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:04:51.502127 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1399/2802] 10.222.0.61 () {58 vars in 1018 bytes} [Mon Aug 22 20:04:51 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 20:04:51.517875 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e54cae-530d-4514-8590-edd9565fce01 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:04:51.536714 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e54cae-530d-4514-8590-edd9565fce01 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:04:51.540647 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c799f97b-1dac-42dc-af71-629d3d007075 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:04:51.541279 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c799f97b-1dac-42dc-af71-629d3d007075 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:04:51.541663 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c799f97b-1dac-42dc-af71-629d3d007075 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:04:51.542050 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c799f97b-1dac-42dc-af71-629d3d007075 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:04:51.557362 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c799f97b-1dac-42dc-af71-629d3d007075 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:04:51.559308 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c799f97b-1dac-42dc-af71-629d3d007075 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:04:51.570840 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1404/2803] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 20:04:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 20:05:01.483573 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=9 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:05:01.565747 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=9 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:05:51.444492 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f814c935-b71d-40b9-956d-f6828ae79133 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:05:51.445163 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f814c935-b71d-40b9-956d-f6828ae79133 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:05:51.445163 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f814c935-b71d-40b9-956d-f6828ae79133 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:05:51.464820 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-f814c935-b71d-40b9-956d-f6828ae79133 None None] MFA Rules not processed for user `e0dc5256349a442db02b6e5584872356`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 20:05:51.466021 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-f814c935-b71d-40b9-956d-f6828ae79133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:05:51.472786 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1400/2804] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 20:05:51 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 20:05:54.897872 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c799f97b-1dac-42dc-af71-629d3d007075 service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:05:54.907197 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c799f97b-1dac-42dc-af71-629d3d007075 service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:05:54.910931 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fb46ac-2cda-4095-9451-f61650d5c962 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:05:54.911537 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fb46ac-2cda-4095-9451-f61650d5c962 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:05:54.911842 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fb46ac-2cda-4095-9451-f61650d5c962 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:05:54.912109 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fb46ac-2cda-4095-9451-f61650d5c962 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:05:54.913871 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-94fb46ac-2cda-4095-9451-f61650d5c962 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:05:54.949675 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fb46ac-2cda-4095-9451-f61650d5c962 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:05:54.951821 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fb46ac-2cda-4095-9451-f61650d5c962 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:05:54.960706 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1405/2805] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 20:05:54 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 20:06:01.459161 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:06:04.956773 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=11 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:06:47.957339 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f814c935-b71d-40b9-956d-f6828ae79133 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:06:47.967424 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f814c935-b71d-40b9-956d-f6828ae79133 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:06:47.971611 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf8ab37-13e1-46a5-847e-b68ccb3b6b49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:06:47.972496 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf8ab37-13e1-46a5-847e-b68ccb3b6b49 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:06:47.972496 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf8ab37-13e1-46a5-847e-b68ccb3b6b49 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:06:47.972764 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf8ab37-13e1-46a5-847e-b68ccb3b6b49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:06:47.974464 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-5bf8ab37-13e1-46a5-847e-b68ccb3b6b49 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:06:48.035611 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf8ab37-13e1-46a5-847e-b68ccb3b6b49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:06:48.037067 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf8ab37-13e1-46a5-847e-b68ccb3b6b49 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:06:48.047579 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1401/2806] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 20:06:47 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 20:06:49.038429 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fb46ac-2cda-4095-9451-f61650d5c962 service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:06:49.047898 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fb46ac-2cda-4095-9451-f61650d5c962 service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:06:49.051451 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f72ad02-89d3-403c-a8aa-605395c1efbc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:06:49.052088 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f72ad02-89d3-403c-a8aa-605395c1efbc service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:06:49.052363 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f72ad02-89d3-403c-a8aa-605395c1efbc service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:06:49.052627 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f72ad02-89d3-403c-a8aa-605395c1efbc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:06:49.054311 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-7f72ad02-89d3-403c-a8aa-605395c1efbc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:06:49.082669 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f72ad02-89d3-403c-a8aa-605395c1efbc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:06:49.084062 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f72ad02-89d3-403c-a8aa-605395c1efbc service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:06:49.095675 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1406/2807] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 20:06:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 20:06:50.279264 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf8ab37-13e1-46a5-847e-b68ccb3b6b49 service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:06:50.288479 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf8ab37-13e1-46a5-847e-b68ccb3b6b49 service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:06:50.292712 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4553201-4685-45a6-a3cd-96027ebbf5fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:06:50.293356 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4553201-4685-45a6-a3cd-96027ebbf5fc service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:06:50.293642 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4553201-4685-45a6-a3cd-96027ebbf5fc service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:06:50.293921 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4553201-4685-45a6-a3cd-96027ebbf5fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:06:50.295750 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-f4553201-4685-45a6-a3cd-96027ebbf5fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:06:50.310617 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4553201-4685-45a6-a3cd-96027ebbf5fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:06:50.312082 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4553201-4685-45a6-a3cd-96027ebbf5fc service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:06:50.323237 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1402/2808] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 20:06:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 20:06:51.506374 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f72ad02-89d3-403c-a8aa-605395c1efbc service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:06:51.519527 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f72ad02-89d3-403c-a8aa-605395c1efbc service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:06:51.525143 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccc2ae2-e486-4d18-b19a-5226658d1336 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f43e2756b746a6825ca1a06b7d7c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:06:51.525981 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccc2ae2-e486-4d18-b19a-5226658d1336 service nova] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:06:51.526254 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccc2ae2-e486-4d18-b19a-5226658d1336 service nova] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:06:51.526690 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccc2ae2-e486-4d18-b19a-5226658d1336 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:06:51.529045 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-6ccc2ae2-e486-4d18-b19a-5226658d1336 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:06:51.542610 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccc2ae2-e486-4d18-b19a-5226658d1336 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:06:51.544374 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccc2ae2-e486-4d18-b19a-5226658d1336 service nova] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:06:51.556791 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1407/2809] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 20:06:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 20:06:52.810089 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4553201-4685-45a6-a3cd-96027ebbf5fc service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:06:52.818746 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4553201-4685-45a6-a3cd-96027ebbf5fc service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:06:52.822360 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eb3aba-6a79-45cc-9fbd-a068d9400b4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:06:52.822955 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb3aba-6a79-45cc-9fbd-a068d9400b4b service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:06:52.823233 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb3aba-6a79-45cc-9fbd-a068d9400b4b service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:06:52.823503 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb3aba-6a79-45cc-9fbd-a068d9400b4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:06:52.825786 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-22eb3aba-6a79-45cc-9fbd-a068d9400b4b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:06:52.859586 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22eb3aba-6a79-45cc-9fbd-a068d9400b4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:06:52.861111 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22eb3aba-6a79-45cc-9fbd-a068d9400b4b service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:06:52.879998 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1403/2810] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 20:06:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 20:06:55.814496 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccc2ae2-e486-4d18-b19a-5226658d1336 service nova] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:06:55.826816 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccc2ae2-e486-4d18-b19a-5226658d1336 service nova] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:06:55.832617 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccea9e3-ef0f-4630-a7da-2e7888695529 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eef0f73297e04d62bc22acb790fad6ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:06:55.833542 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccea9e3-ef0f-4630-a7da-2e7888695529 service placement] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:06:55.833884 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccea9e3-ef0f-4630-a7da-2e7888695529 service placement] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:06:55.834231 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccea9e3-ef0f-4630-a7da-2e7888695529 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:06:55.838594 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-9ccea9e3-ef0f-4630-a7da-2e7888695529 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:06:55.851238 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccea9e3-ef0f-4630-a7da-2e7888695529 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:06:55.852943 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccea9e3-ef0f-4630-a7da-2e7888695529 service placement] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:06:55.864292 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1408/2811] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 20:06:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 20:06:58.895883 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eb3aba-6a79-45cc-9fbd-a068d9400b4b service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:06:58.910551 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eb3aba-6a79-45cc-9fbd-a068d9400b4b service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:06:58.920663 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74deab0f-5452-4fa8-9d4c-1c415731f1cc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eef0f73297e04d62bc22acb790fad6ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:06:58.921583 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74deab0f-5452-4fa8-9d4c-1c415731f1cc service placement] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:06:58.922019 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74deab0f-5452-4fa8-9d4c-1c415731f1cc service placement] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:06:58.922466 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74deab0f-5452-4fa8-9d4c-1c415731f1cc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:06:58.924646 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-74deab0f-5452-4fa8-9d4c-1c415731f1cc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:06:58.939327 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74deab0f-5452-4fa8-9d4c-1c415731f1cc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:06:58.941083 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74deab0f-5452-4fa8-9d4c-1c415731f1cc service placement] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:06:58.953868 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1404/2812] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 20:06:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 20:06:58.967174 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccea9e3-ef0f-4630-a7da-2e7888695529 service placement] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:06:58.969605 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-9ccea9e3-ef0f-4630-a7da-2e7888695529 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:06:58.987244 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccea9e3-ef0f-4630-a7da-2e7888695529 service placement] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:06:58.993334 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c527fac9-312a-496d-944d-0c0650dd6005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:06:58.994285 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c527fac9-312a-496d-944d-0c0650dd6005 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:06:58.994685 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c527fac9-312a-496d-944d-0c0650dd6005 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:06:58.994962 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c527fac9-312a-496d-944d-0c0650dd6005 admin admin] PATH_INFO: `/v3/users/ae3afb9ab5c3485287cc352735a159b8` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:06:58.996238 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c527fac9-312a-496d-944d-0c0650dd6005 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae3afb9ab5c3485287cc352735a159b8)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:06:58.998128 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c527fac9-312a-496d-944d-0c0650dd6005 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:06:59.047154 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-c527fac9-312a-496d-944d-0c0650dd6005 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae3afb9ab5c3485287cc352735a159b8'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:06:59.047762 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-c527fac9-312a-496d-944d-0c0650dd6005 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae3afb9ab5c3485287cc352735a159b8'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:06:59.054639 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-c527fac9-312a-496d-944d-0c0650dd6005 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae3afb9ab5c3485287cc352735a159b8'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:06:59.063529 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-c527fac9-312a-496d-944d-0c0650dd6005 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae3afb9ab5c3485287cc352735a159b8'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:06:59.080491 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-c527fac9-312a-496d-944d-0c0650dd6005 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae3afb9ab5c3485287cc352735a159b8'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:06:59.089004 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1409/2813] 10.222.0.61 () {58 vars in 1224 bytes} [Mon Aug 22 20:06:58 2022] DELETE /identity/v3/users/ae3afb9ab5c3485287cc352735a159b8 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 20:06:59.268354 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74deab0f-5452-4fa8-9d4c-1c415731f1cc service placement] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:06:59.280674 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74deab0f-5452-4fa8-9d4c-1c415731f1cc service placement] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:06:59.336725 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cbd28e-0069-49cd-b199-3147fd546d07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:06:59.337355 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cbd28e-0069-49cd-b199-3147fd546d07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:06:59.337630 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cbd28e-0069-49cd-b199-3147fd546d07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:06:59.337902 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cbd28e-0069-49cd-b199-3147fd546d07 admin admin] PATH_INFO: `/v3/projects/e769d7a29dbc470f9013344d9c817d63` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:06:59.339046 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cbd28e-0069-49cd-b199-3147fd546d07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e769d7a29dbc470f9013344d9c817d63)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:06:59.341506 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cbd28e-0069-49cd-b199-3147fd546d07 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:06:59.375476 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:06:59.389314 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-16cbd28e-0069-49cd-b199-3147fd546d07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e769d7a29dbc470f9013344d9c817d63', 'request_id': 'req-16cbd28e-0069-49cd-b199-3147fd546d07'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:06:59.396161 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-16cbd28e-0069-49cd-b199-3147fd546d07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e769d7a29dbc470f9013344d9c817d63', 'request_id': 'req-16cbd28e-0069-49cd-b199-3147fd546d07'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:06:59.404904 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-16cbd28e-0069-49cd-b199-3147fd546d07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e769d7a29dbc470f9013344d9c817d63', 'request_id': 'req-16cbd28e-0069-49cd-b199-3147fd546d07'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:06:59.415375 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-16cbd28e-0069-49cd-b199-3147fd546d07 admin admin] The token cache is being invalidate because project e769d7a29dbc470f9013344d9c817d63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 20:06:59.415684 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-16cbd28e-0069-49cd-b199-3147fd546d07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:06:59.416364 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-16cbd28e-0069-49cd-b199-3147fd546d07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:06:59.417869 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1405/2814] 10.222.0.61 () {58 vars in 1233 bytes} [Mon Aug 22 20:06:59 2022] DELETE /identity/v3/projects/e769d7a29dbc470f9013344d9c817d63 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 20:06:59.421847 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c527fac9-312a-496d-944d-0c0650dd6005 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:06:59.423235 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-c527fac9-312a-496d-944d-0c0650dd6005 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:06:59.436317 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c527fac9-312a-496d-944d-0c0650dd6005 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:06:59.495462 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1c3a10-4a06-45bc-a34f-66a4ef2e014b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:06:59.496042 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1c3a10-4a06-45bc-a34f-66a4ef2e014b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:06:59.496320 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1c3a10-4a06-45bc-a34f-66a4ef2e014b admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:06:59.496595 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1c3a10-4a06-45bc-a34f-66a4ef2e014b admin admin] PATH_INFO: `/v3/users/6ac28b13543d4dc98178965372ef765d` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:06:59.497771 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1c3a10-4a06-45bc-a34f-66a4ef2e014b admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ac28b13543d4dc98178965372ef765d)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:06:59.499750 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1c3a10-4a06-45bc-a34f-66a4ef2e014b admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:06:59.522448 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:06:59.549032 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-4e1c3a10-4a06-45bc-a34f-66a4ef2e014b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ac28b13543d4dc98178965372ef765d'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:06:59.549678 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-4e1c3a10-4a06-45bc-a34f-66a4ef2e014b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ac28b13543d4dc98178965372ef765d'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:06:59.556503 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-4e1c3a10-4a06-45bc-a34f-66a4ef2e014b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ac28b13543d4dc98178965372ef765d'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:06:59.563564 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-4e1c3a10-4a06-45bc-a34f-66a4ef2e014b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ac28b13543d4dc98178965372ef765d'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:06:59.576085 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-4e1c3a10-4a06-45bc-a34f-66a4ef2e014b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ac28b13543d4dc98178965372ef765d'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:06:59.581823 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1410/2815] 10.222.0.61 () {58 vars in 1224 bytes} [Mon Aug 22 20:06:59 2022] DELETE /identity/v3/users/6ac28b13543d4dc98178965372ef765d => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 20:06:59.756241 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cbd28e-0069-49cd-b199-3147fd546d07 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:06:59.757823 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-16cbd28e-0069-49cd-b199-3147fd546d07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:06:59.772683 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cbd28e-0069-49cd-b199-3147fd546d07 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:06:59.830261 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d825493-b921-420c-9951-6ec6be943395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:06:59.830889 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d825493-b921-420c-9951-6ec6be943395 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:06:59.831167 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d825493-b921-420c-9951-6ec6be943395 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:06:59.831431 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d825493-b921-420c-9951-6ec6be943395 admin admin] PATH_INFO: `/v3/projects/299cd3a442a34d1fa79f4ccce35b3fb4` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:06:59.832576 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d825493-b921-420c-9951-6ec6be943395 admin admin] RBAC: Authorizing `identity:delete_project(project_id=299cd3a442a34d1fa79f4ccce35b3fb4)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:06:59.834451 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d825493-b921-420c-9951-6ec6be943395 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:06:59.879389 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-0d825493-b921-420c-9951-6ec6be943395 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '299cd3a442a34d1fa79f4ccce35b3fb4', 'request_id': 'req-0d825493-b921-420c-9951-6ec6be943395'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:06:59.887397 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-0d825493-b921-420c-9951-6ec6be943395 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '299cd3a442a34d1fa79f4ccce35b3fb4', 'request_id': 'req-0d825493-b921-420c-9951-6ec6be943395'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:06:59.893968 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-0d825493-b921-420c-9951-6ec6be943395 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '299cd3a442a34d1fa79f4ccce35b3fb4', 'request_id': 'req-0d825493-b921-420c-9951-6ec6be943395'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:06:59.904397 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-0d825493-b921-420c-9951-6ec6be943395 admin admin] The token cache is being invalidate because project 299cd3a442a34d1fa79f4ccce35b3fb4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 20:06:59.905033 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-0d825493-b921-420c-9951-6ec6be943395 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:06:59.905371 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-0d825493-b921-420c-9951-6ec6be943395 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:06:59.906876 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1406/2816] 10.222.0.61 () {58 vars in 1233 bytes} [Mon Aug 22 20:06:59 2022] DELETE /identity/v3/projects/299cd3a442a34d1fa79f4ccce35b3fb4 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 20:07:09.578521 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=20,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:07:09.901362 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=63,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:07:20.007039 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1c3a10-4a06-45bc-a34f-66a4ef2e014b admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:07:20.008500 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-4e1c3a10-4a06-45bc-a34f-66a4ef2e014b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:07:20.081281 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1c3a10-4a06-45bc-a34f-66a4ef2e014b admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:07:20.143663 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad4dc54-c776-4de4-bbd5-e2c6ba4a2f14 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f43e2756b746a6825ca1a06b7d7c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:07:20.144287 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad4dc54-c776-4de4-bbd5-e2c6ba4a2f14 service nova] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:07:20.144569 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad4dc54-c776-4de4-bbd5-e2c6ba4a2f14 service nova] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:07:20.144844 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad4dc54-c776-4de4-bbd5-e2c6ba4a2f14 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:07:20.146631 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-fad4dc54-c776-4de4-bbd5-e2c6ba4a2f14 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:07:20.203753 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad4dc54-c776-4de4-bbd5-e2c6ba4a2f14 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:07:20.205451 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad4dc54-c776-4de4-bbd5-e2c6ba4a2f14 service nova] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:07:20.214145 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1411/2817] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 20:07:20 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 20:07:30.210180 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=84 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:07:37.286955 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d825493-b921-420c-9951-6ec6be943395 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:07:37.288420 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-0d825493-b921-420c-9951-6ec6be943395 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:07:37.305559 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d825493-b921-420c-9951-6ec6be943395 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:07:37.311030 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1263f03b-a5c4-416d-8e89-2a66594fbdac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f43e2756b746a6825ca1a06b7d7c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:07:37.311631 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1263f03b-a5c4-416d-8e89-2a66594fbdac service nova] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:07:37.311941 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1263f03b-a5c4-416d-8e89-2a66594fbdac service nova] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:07:37.312208 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1263f03b-a5c4-416d-8e89-2a66594fbdac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:07:37.313914 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-1263f03b-a5c4-416d-8e89-2a66594fbdac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:07:37.327296 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1263f03b-a5c4-416d-8e89-2a66594fbdac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:07:37.328913 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1263f03b-a5c4-416d-8e89-2a66594fbdac service nova] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:07:37.340103 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1407/2818] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 20:07:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 20:07:39.404229 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787323fe-c553-4747-bba0-cb85b988965f None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:07:39.404722 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787323fe-c553-4747-bba0-cb85b988965f None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:07:39.404824 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787323fe-c553-4747-bba0-cb85b988965f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:07:39.434356 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-787323fe-c553-4747-bba0-cb85b988965f None None] MFA Rules not processed for user `399396a8fe254da199bc48b0ca5243d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 20:07:39.435565 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-787323fe-c553-4747-bba0-cb85b988965f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:07:39.492632 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1412/2819] 10.222.0.61 () {58 vars in 1009 bytes} [Mon Aug 22 20:07:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 20:07:39.540371 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1263f03b-a5c4-416d-8e89-2a66594fbdac service nova] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:07:39.542042 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-1263f03b-a5c4-416d-8e89-2a66594fbdac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:07:39.609123 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1263f03b-a5c4-416d-8e89-2a66594fbdac service nova] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:07:39.614098 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a67b17e-9c30-4b73-9836-0e32a5137b5b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2824359491934d0dacaa21a718becee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:07:39.614704 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a67b17e-9c30-4b73-9836-0e32a5137b5b service swift] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:07:39.614982 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a67b17e-9c30-4b73-9836-0e32a5137b5b service swift] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:07:39.615252 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a67b17e-9c30-4b73-9836-0e32a5137b5b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:07:39.623848 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a67b17e-9c30-4b73-9836-0e32a5137b5b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:07:39.625427 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a67b17e-9c30-4b73-9836-0e32a5137b5b service swift] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:07:39.633553 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1408/2820] 10.222.0.61 () {60 vars in 1384 bytes} [Mon Aug 22 20:07:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 22 20:07:49.484701 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=29 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:07:49.630378 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=38 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:07:53.953366 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bf6487-c0d5-4da6-afff-ec249d9d565f None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:07:53.953922 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bf6487-c0d5-4da6-afff-ec249d9d565f None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:07:53.953922 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bf6487-c0d5-4da6-afff-ec249d9d565f None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:07:53.955322 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1413/2821] 10.222.0.61 () {54 vars in 921 bytes} [Mon Aug 22 20:07:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 20:07:53.961430 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070ef212-da39-4b04-8208-12783182f58c None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:07:53.961903 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070ef212-da39-4b04-8208-12783182f58c None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:07:53.962010 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070ef212-da39-4b04-8208-12783182f58c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:07:53.992770 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-070ef212-da39-4b04-8208-12783182f58c None None] MFA Rules not processed for user `e7f43e2756b746a6825ca1a06b7d7c91`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 20:07:53.993924 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-070ef212-da39-4b04-8208-12783182f58c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:07:54.005220 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1409/2822] 10.222.0.61 () {58 vars in 1018 bytes} [Mon Aug 22 20:07:53 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 20:07:54.017043 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bf6487-c0d5-4da6-afff-ec249d9d565f None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:07:54.018804 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-c2bf6487-c0d5-4da6-afff-ec249d9d565f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:07:54.080043 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bf6487-c0d5-4da6-afff-ec249d9d565f None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:07:54.084452 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a39adc4-d370-424b-9a8f-c2cb95477923 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d6a79ed44e5396d13cd29ba160ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:07:54.085053 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a39adc4-d370-424b-9a8f-c2cb95477923 service glance] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:07:54.085325 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a39adc4-d370-424b-9a8f-c2cb95477923 service glance] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:07:54.085595 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a39adc4-d370-424b-9a8f-c2cb95477923 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:07:54.097004 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a39adc4-d370-424b-9a8f-c2cb95477923 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:07:54.098410 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a39adc4-d370-424b-9a8f-c2cb95477923 service glance] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:07:54.109184 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1414/2823] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 20:07:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 20:07:54.295027 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070ef212-da39-4b04-8208-12783182f58c None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:07:54.296485 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-070ef212-da39-4b04-8208-12783182f58c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:07:54.363887 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070ef212-da39-4b04-8208-12783182f58c None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:07:54.368836 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3c35f6-7df5-43f0-b8d6-6e0c6aa07953 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eef0f73297e04d62bc22acb790fad6ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:07:54.370023 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3c35f6-7df5-43f0-b8d6-6e0c6aa07953 service placement] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:07:54.370509 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3c35f6-7df5-43f0-b8d6-6e0c6aa07953 service placement] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:07:54.370969 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3c35f6-7df5-43f0-b8d6-6e0c6aa07953 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:07:54.373869 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-ef3c35f6-7df5-43f0-b8d6-6e0c6aa07953 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:07:54.386565 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3c35f6-7df5-43f0-b8d6-6e0c6aa07953 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:07:54.388161 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3c35f6-7df5-43f0-b8d6-6e0c6aa07953 service placement] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:07:54.397994 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1410/2824] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 20:07:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 20:07:56.000514 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e184c8ee-f702-490c-b759-5bf5d1874926 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:07:56.001019 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e184c8ee-f702-490c-b759-5bf5d1874926 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:07:56.001132 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e184c8ee-f702-490c-b759-5bf5d1874926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:07:56.020205 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-e184c8ee-f702-490c-b759-5bf5d1874926 None None] MFA Rules not processed for user `399396a8fe254da199bc48b0ca5243d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 20:07:56.021404 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-e184c8ee-f702-490c-b759-5bf5d1874926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:07:56.037653 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1415/2825] 10.222.0.61 () {58 vars in 1009 bytes} [Mon Aug 22 20:07:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 20:07:56.049187 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3c35f6-7df5-43f0-b8d6-6e0c6aa07953 service placement] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:07:56.081451 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3c35f6-7df5-43f0-b8d6-6e0c6aa07953 service placement] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:07:56.086409 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a3e9fd-be68-4ae3-a7a3-69a865356293 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2824359491934d0dacaa21a718becee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:07:56.086999 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a3e9fd-be68-4ae3-a7a3-69a865356293 service swift] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:07:56.087276 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a3e9fd-be68-4ae3-a7a3-69a865356293 service swift] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:07:56.087550 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a3e9fd-be68-4ae3-a7a3-69a865356293 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:07:56.095489 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a3e9fd-be68-4ae3-a7a3-69a865356293 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:07:56.097075 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a3e9fd-be68-4ae3-a7a3-69a865356293 service swift] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:07:56.105707 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1411/2826] 10.222.0.61 () {60 vars in 1384 bytes} [Mon Aug 22 20:07:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 22 20:08:06.029878 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=33 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:08:06.101858 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=54 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:08:09.439696 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e184c8ee-f702-490c-b759-5bf5d1874926 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:08:09.469098 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e184c8ee-f702-490c-b759-5bf5d1874926 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:08:09.477025 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548ad0f9-a77e-4fae-a7cd-85cef47892d5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eef0f73297e04d62bc22acb790fad6ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:08:09.477661 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548ad0f9-a77e-4fae-a7cd-85cef47892d5 service placement] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:08:09.477938 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548ad0f9-a77e-4fae-a7cd-85cef47892d5 service placement] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:08:09.478208 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548ad0f9-a77e-4fae-a7cd-85cef47892d5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:08:09.488820 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-548ad0f9-a77e-4fae-a7cd-85cef47892d5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:08:09.524142 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548ad0f9-a77e-4fae-a7cd-85cef47892d5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:08:09.525841 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548ad0f9-a77e-4fae-a7cd-85cef47892d5 service placement] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:08:09.536557 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1416/2827] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 20:08:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 20:08:13.654340 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a55203-be85-43d6-899a-b07ac1a28569 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:08:13.655457 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a55203-be85-43d6-899a-b07ac1a28569 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:08:13.655457 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a55203-be85-43d6-899a-b07ac1a28569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:08:13.677025 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-34a55203-be85-43d6-899a-b07ac1a28569 None None] MFA Rules not processed for user `399396a8fe254da199bc48b0ca5243d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 20:08:13.678222 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-34a55203-be85-43d6-899a-b07ac1a28569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:08:13.685605 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1412/2828] 10.222.0.61 () {58 vars in 967 bytes} [Mon Aug 22 20:08:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 20:08:13.697611 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548ad0f9-a77e-4fae-a7cd-85cef47892d5 service placement] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:08:13.709286 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548ad0f9-a77e-4fae-a7cd-85cef47892d5 service placement] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:08:13.714213 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070a1fdb-a7b8-45b7-8ed1-a4e1324288ac service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2824359491934d0dacaa21a718becee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:08:13.714810 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070a1fdb-a7b8-45b7-8ed1-a4e1324288ac service swift] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:08:13.715085 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070a1fdb-a7b8-45b7-8ed1-a4e1324288ac service swift] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:08:13.715358 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070a1fdb-a7b8-45b7-8ed1-a4e1324288ac service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:08:13.724259 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070a1fdb-a7b8-45b7-8ed1-a4e1324288ac service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:08:13.725862 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070a1fdb-a7b8-45b7-8ed1-a4e1324288ac service swift] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:08:13.734502 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1417/2829] 10.222.0.61 () {60 vars in 1384 bytes} [Mon Aug 22 20:08:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 22 20:08:23.672277 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:08:23.730594 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=21 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:09:17.750678 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dd7a8e-3acd-422b-9b59-c64f6b4862b9 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:17.751119 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dd7a8e-3acd-422b-9b59-c64f6b4862b9 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:17.751244 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dd7a8e-3acd-422b-9b59-c64f6b4862b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:17.770816 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-f6dd7a8e-3acd-422b-9b59-c64f6b4862b9 None None] MFA Rules not processed for user `a4aca06c966a42f29f5c5bc108529d41`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 20:09:17.772053 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-f6dd7a8e-3acd-422b-9b59-c64f6b4862b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:09:17.824530 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1413/2830] 10.222.0.61 () {58 vars in 1062 bytes} [Mon Aug 22 20:09:17 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 20:09:17.830979 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070a1fdb-a7b8-45b7-8ed1-a4e1324288ac service swift] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:17.840271 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070a1fdb-a7b8-45b7-8ed1-a4e1324288ac service swift] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:17.844477 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb761360-d03d-46a6-bfb7-7fbe2972da04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:17.845084 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb761360-d03d-46a6-bfb7-7fbe2972da04 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:17.845343 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb761360-d03d-46a6-bfb7-7fbe2972da04 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:17.845615 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb761360-d03d-46a6-bfb7-7fbe2972da04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:17.847327 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-eb761360-d03d-46a6-bfb7-7fbe2972da04 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:09:17.909134 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb761360-d03d-46a6-bfb7-7fbe2972da04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:17.910477 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb761360-d03d-46a6-bfb7-7fbe2972da04 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:17.919781 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1418/2831] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 20:09:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 20:09:20.758357 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dd7a8e-3acd-422b-9b59-c64f6b4862b9 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:20.760126 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-f6dd7a8e-3acd-422b-9b59-c64f6b4862b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:09:20.777531 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dd7a8e-3acd-422b-9b59-c64f6b4862b9 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:20.782337 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac0d2f3-212d-407f-95ed-763c40bcddb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:20.783034 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac0d2f3-212d-407f-95ed-763c40bcddb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:20.783312 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac0d2f3-212d-407f-95ed-763c40bcddb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:20.783582 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac0d2f3-212d-407f-95ed-763c40bcddb3 admin admin] PATH_INFO: `/v3/users/e0dc5256349a442db02b6e5584872356` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:20.784867 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac0d2f3-212d-407f-95ed-763c40bcddb3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0dc5256349a442db02b6e5584872356)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:20.786833 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac0d2f3-212d-407f-95ed-763c40bcddb3 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:20.841114 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-1ac0d2f3-212d-407f-95ed-763c40bcddb3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0dc5256349a442db02b6e5584872356'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:20.841686 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-1ac0d2f3-212d-407f-95ed-763c40bcddb3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0dc5256349a442db02b6e5584872356'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:20.849035 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-1ac0d2f3-212d-407f-95ed-763c40bcddb3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0dc5256349a442db02b6e5584872356'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:20.856221 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-1ac0d2f3-212d-407f-95ed-763c40bcddb3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0dc5256349a442db02b6e5584872356'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:20.869771 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-1ac0d2f3-212d-407f-95ed-763c40bcddb3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0dc5256349a442db02b6e5584872356'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:20.876440 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1414/2832] 10.222.0.61 () {58 vars in 1224 bytes} [Mon Aug 22 20:09:20 2022] DELETE /identity/v3/users/e0dc5256349a442db02b6e5584872356 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 20:09:21.050168 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb761360-d03d-46a6-bfb7-7fbe2972da04 service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:21.062227 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb761360-d03d-46a6-bfb7-7fbe2972da04 service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:21.121751 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e468044-9a90-4989-85b0-a292d8efa5cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:21.122312 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e468044-9a90-4989-85b0-a292d8efa5cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:21.122588 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e468044-9a90-4989-85b0-a292d8efa5cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:21.122858 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e468044-9a90-4989-85b0-a292d8efa5cf admin admin] PATH_INFO: `/v3/projects/548a965fed81436fa0440d4c4db00e2c` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:21.124015 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e468044-9a90-4989-85b0-a292d8efa5cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=548a965fed81436fa0440d4c4db00e2c)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:21.125920 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e468044-9a90-4989-85b0-a292d8efa5cf admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:21.172202 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-6e468044-9a90-4989-85b0-a292d8efa5cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '548a965fed81436fa0440d4c4db00e2c', 'request_id': 'req-6e468044-9a90-4989-85b0-a292d8efa5cf'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:21.179476 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-6e468044-9a90-4989-85b0-a292d8efa5cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '548a965fed81436fa0440d4c4db00e2c', 'request_id': 'req-6e468044-9a90-4989-85b0-a292d8efa5cf'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:21.185559 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-6e468044-9a90-4989-85b0-a292d8efa5cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '548a965fed81436fa0440d4c4db00e2c', 'request_id': 'req-6e468044-9a90-4989-85b0-a292d8efa5cf'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:21.196147 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-6e468044-9a90-4989-85b0-a292d8efa5cf admin admin] The token cache is being invalidate because project 548a965fed81436fa0440d4c4db00e2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 20:09:21.196439 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-6e468044-9a90-4989-85b0-a292d8efa5cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:21.197359 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-6e468044-9a90-4989-85b0-a292d8efa5cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:21.198842 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1419/2833] 10.222.0.61 () {58 vars in 1233 bytes} [Mon Aug 22 20:09:21 2022] DELETE /identity/v3/projects/548a965fed81436fa0440d4c4db00e2c => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 20:09:21.203952 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad332635-f889-4ff5-936a-749e7dd017d4 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:21.204238 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad332635-f889-4ff5-936a-749e7dd017d4 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:21.204509 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad332635-f889-4ff5-936a-749e7dd017d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:21.252229 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-ad332635-f889-4ff5-936a-749e7dd017d4 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 20:09:21.253481 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-ad332635-f889-4ff5-936a-749e7dd017d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:09:21.303026 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:09:21.312969 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1415/2834] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 20:09:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 20:09:21.320804 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad736be-4fb6-4872-9ccf-edea2e261476 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:21.321082 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad736be-4fb6-4872-9ccf-edea2e261476 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:21.321351 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad736be-4fb6-4872-9ccf-edea2e261476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:21.340852 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-9ad736be-4fb6-4872-9ccf-edea2e261476 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 20:09:21.342043 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-9ad736be-4fb6-4872-9ccf-edea2e261476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:09:21.350319 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1420/2835] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 20:09:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 20:09:21.354417 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad332635-f889-4ff5-936a-749e7dd017d4 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:21.364732 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad332635-f889-4ff5-936a-749e7dd017d4 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:21.369699 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bba19f1-9565-49de-a3f9-a73b543679f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:21.370289 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bba19f1-9565-49de-a3f9-a73b543679f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:21.370566 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bba19f1-9565-49de-a3f9-a73b543679f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:21.370836 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bba19f1-9565-49de-a3f9-a73b543679f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:21.371481 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bba19f1-9565-49de-a3f9-a73b543679f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 20:09:21.371820 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bba19f1-9565-49de-a3f9-a73b543679f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:21.373599 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bba19f1-9565-49de-a3f9-a73b543679f3 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:21.384913 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1416/2836] 10.222.0.61 () {58 vars in 1153 bytes} [Mon Aug 22 20:09:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 20:09:21.389126 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad736be-4fb6-4872-9ccf-edea2e261476 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:21.398859 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad736be-4fb6-4872-9ccf-edea2e261476 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:21.403625 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6858cd0f-32ee-4fa6-b49e-89cecf0ebbef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:21.404277 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6858cd0f-32ee-4fa6-b49e-89cecf0ebbef admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:21.404555 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6858cd0f-32ee-4fa6-b49e-89cecf0ebbef admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:21.404829 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6858cd0f-32ee-4fa6-b49e-89cecf0ebbef admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:21.406086 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6858cd0f-32ee-4fa6-b49e-89cecf0ebbef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:21.408004 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6858cd0f-32ee-4fa6-b49e-89cecf0ebbef admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:21.423006 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:09:21.445514 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1421/2837] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 20:09:21 2022] POST /identity/v3/projects => generated 387 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 20:09:21.449908 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bba19f1-9565-49de-a3f9-a73b543679f3 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:21.460339 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bba19f1-9565-49de-a3f9-a73b543679f3 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:21.515707 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3736399-09f9-4e80-9d7a-2d9879b03b7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:21.516301 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3736399-09f9-4e80-9d7a-2d9879b03b7b admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:21.516577 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3736399-09f9-4e80-9d7a-2d9879b03b7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:21.516845 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3736399-09f9-4e80-9d7a-2d9879b03b7b admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:21.517469 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3736399-09f9-4e80-9d7a-2d9879b03b7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:21.519440 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3736399-09f9-4e80-9d7a-2d9879b03b7b admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:21.539989 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1417/2838] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 20:09:21 2022] POST /identity/v3/users => generated 294 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 20:09:21.544076 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6858cd0f-32ee-4fa6-b49e-89cecf0ebbef admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:21.554387 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6858cd0f-32ee-4fa6-b49e-89cecf0ebbef admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:21.558890 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75b3d7e-fb64-44cf-8970-099e3eb62a5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:21.559479 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75b3d7e-fb64-44cf-8970-099e3eb62a5d admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:21.559799 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75b3d7e-fb64-44cf-8970-099e3eb62a5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:21.560070 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75b3d7e-fb64-44cf-8970-099e3eb62a5d admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:21.560740 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75b3d7e-fb64-44cf-8970-099e3eb62a5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:21.562330 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75b3d7e-fb64-44cf-8970-099e3eb62a5d admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:21.572802 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1422/2839] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 20:09:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 20:09:21.576563 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3736399-09f9-4e80-9d7a-2d9879b03b7b admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:21.587103 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3736399-09f9-4e80-9d7a-2d9879b03b7b admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:21.591785 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94d7fe9-5b47-433b-937a-464c949ec17f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:21.592391 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94d7fe9-5b47-433b-937a-464c949ec17f admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:21.592668 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94d7fe9-5b47-433b-937a-464c949ec17f admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:21.593014 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94d7fe9-5b47-433b-937a-464c949ec17f admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:21.594861 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94d7fe9-5b47-433b-937a-464c949ec17f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:21.596478 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94d7fe9-5b47-433b-937a-464c949ec17f admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:21.606839 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1418/2840] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 20:09:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 20:09:21.610941 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75b3d7e-fb64-44cf-8970-099e3eb62a5d admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:21.621147 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75b3d7e-fb64-44cf-8970-099e3eb62a5d admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:21.625811 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f557e09a-9fac-4da9-a587-a172473fc057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:21.626491 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f557e09a-9fac-4da9-a587-a172473fc057 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:21.626760 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f557e09a-9fac-4da9-a587-a172473fc057 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:21.627023 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f557e09a-9fac-4da9-a587-a172473fc057 admin admin] PATH_INFO: `/v3/projects/c857a712aed545ce97e8a74b893c0fa0/users/041f110de3cc4dc3b4f2850a6506e69d/roles/74a40bc819cb42eea7e966a4b2a016f1` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:21.642150 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f557e09a-9fac-4da9-a587-a172473fc057 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c857a712aed545ce97e8a74b893c0fa0, user_id=041f110de3cc4dc3b4f2850a6506e69d, role_id=74a40bc819cb42eea7e966a4b2a016f1)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:21.644811 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f557e09a-9fac-4da9-a587-a172473fc057 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:21.660221 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1423/2841] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 20:09:21 2022] PUT /identity/v3/projects/c857a712aed545ce97e8a74b893c0fa0/users/041f110de3cc4dc3b4f2850a6506e69d/roles/74a40bc819cb42eea7e966a4b2a016f1 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 20:09:21.671553 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464f3af9-3eab-44d5-96a7-63229a9afc26 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:21.671876 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464f3af9-3eab-44d5-96a7-63229a9afc26 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:21.672151 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464f3af9-3eab-44d5-96a7-63229a9afc26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:21.698941 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-464f3af9-3eab-44d5-96a7-63229a9afc26 None None] MFA Rules not processed for user `041f110de3cc4dc3b4f2850a6506e69d`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 20:09:21.700195 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-464f3af9-3eab-44d5-96a7-63229a9afc26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:09:21.763020 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1419/2842] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 20:09:21 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 20:09:21.768336 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f557e09a-9fac-4da9-a587-a172473fc057 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:21.829029 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f557e09a-9fac-4da9-a587-a172473fc057 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:21.833583 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cffb4d-ac83-4afe-89f0-9fa65600d1f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:21.834184 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cffb4d-ac83-4afe-89f0-9fa65600d1f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:21.834461 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cffb4d-ac83-4afe-89f0-9fa65600d1f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:21.834732 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cffb4d-ac83-4afe-89f0-9fa65600d1f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:21.835348 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cffb4d-ac83-4afe-89f0-9fa65600d1f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:21.837280 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cffb4d-ac83-4afe-89f0-9fa65600d1f7 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:21.862294 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1424/2843] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 20:09:21 2022] POST /identity/v3/projects => generated 385 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 20:09:21.866395 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464f3af9-3eab-44d5-96a7-63229a9afc26 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:21.877922 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464f3af9-3eab-44d5-96a7-63229a9afc26 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:21.927698 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:09:21.936038 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c57cafd-ce0f-4e2d-ac7a-ccd587ce41da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:21.937257 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c57cafd-ce0f-4e2d-ac7a-ccd587ce41da admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:21.937534 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c57cafd-ce0f-4e2d-ac7a-ccd587ce41da admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:21.937810 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c57cafd-ce0f-4e2d-ac7a-ccd587ce41da admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:21.938455 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c57cafd-ce0f-4e2d-ac7a-ccd587ce41da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:21.940398 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c57cafd-ce0f-4e2d-ac7a-ccd587ce41da admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:21.962034 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1420/2844] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 20:09:21 2022] POST /identity/v3/users => generated 299 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 20:09:21.965966 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cffb4d-ac83-4afe-89f0-9fa65600d1f7 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:21.976028 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cffb4d-ac83-4afe-89f0-9fa65600d1f7 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:21.980549 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade37769-1946-418c-831a-a49f004ae756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:21.981136 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade37769-1946-418c-831a-a49f004ae756 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:21.981411 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade37769-1946-418c-831a-a49f004ae756 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:21.981677 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade37769-1946-418c-831a-a49f004ae756 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:21.982307 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade37769-1946-418c-831a-a49f004ae756 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:21.983905 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade37769-1946-418c-831a-a49f004ae756 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:21.995336 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1425/2845] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 20:09:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 20:09:21.999825 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c57cafd-ce0f-4e2d-ac7a-ccd587ce41da admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:22.009536 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c57cafd-ce0f-4e2d-ac7a-ccd587ce41da admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:22.014024 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee650fb5-33f4-4581-b7fd-146711eb81bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:22.014653 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee650fb5-33f4-4581-b7fd-146711eb81bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:22.015516 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee650fb5-33f4-4581-b7fd-146711eb81bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:22.015823 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee650fb5-33f4-4581-b7fd-146711eb81bc admin admin] PATH_INFO: `/v3/projects/82962266040f437a9a0c5f6132dad0d9/users/144f9502176d4a18a578e9f4d1739ad0/roles/163c30c3b3c246beb849ce176f02d9da` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:22.029962 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee650fb5-33f4-4581-b7fd-146711eb81bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=82962266040f437a9a0c5f6132dad0d9, user_id=144f9502176d4a18a578e9f4d1739ad0, role_id=163c30c3b3c246beb849ce176f02d9da)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:22.032662 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee650fb5-33f4-4581-b7fd-146711eb81bc admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:22.044611 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1421/2846] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 20:09:21 2022] PUT /identity/v3/projects/82962266040f437a9a0c5f6132dad0d9/users/144f9502176d4a18a578e9f4d1739ad0/roles/163c30c3b3c246beb849ce176f02d9da => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 20:09:22.048452 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade37769-1946-418c-831a-a49f004ae756 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:22.058113 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade37769-1946-418c-831a-a49f004ae756 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:22.116780 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd353a63-5adc-4630-90de-3eb266147ee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:22.117322 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd353a63-5adc-4630-90de-3eb266147ee9 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:22.117601 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd353a63-5adc-4630-90de-3eb266147ee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:22.117881 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd353a63-5adc-4630-90de-3eb266147ee9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:22.118524 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd353a63-5adc-4630-90de-3eb266147ee9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:22.120160 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd353a63-5adc-4630-90de-3eb266147ee9 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:22.131040 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1426/2847] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 20:09:22 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 20:09:22.135265 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee650fb5-33f4-4581-b7fd-146711eb81bc admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:22.146169 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee650fb5-33f4-4581-b7fd-146711eb81bc admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:22.150889 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9b14ba-9969-42c2-95f2-c0a1039c9ba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:22.151537 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9b14ba-9969-42c2-95f2-c0a1039c9ba9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:22.151847 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9b14ba-9969-42c2-95f2-c0a1039c9ba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:22.152119 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9b14ba-9969-42c2-95f2-c0a1039c9ba9 admin admin] PATH_INFO: `/v3/projects/82962266040f437a9a0c5f6132dad0d9/users/144f9502176d4a18a578e9f4d1739ad0/roles/163c30c3b3c246beb849ce176f02d9da` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:22.154032 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9b14ba-9969-42c2-95f2-c0a1039c9ba9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82962266040f437a9a0c5f6132dad0d9, user_id=144f9502176d4a18a578e9f4d1739ad0, role_id=163c30c3b3c246beb849ce176f02d9da)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:22.156891 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9b14ba-9969-42c2-95f2-c0a1039c9ba9 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:22.167101 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1422/2848] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 20:09:22 2022] PUT /identity/v3/projects/82962266040f437a9a0c5f6132dad0d9/users/144f9502176d4a18a578e9f4d1739ad0/roles/163c30c3b3c246beb849ce176f02d9da => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 20:09:22.177883 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faad75d-f693-41fb-bdfc-a1f73dc37cb6 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:22.178249 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faad75d-f693-41fb-bdfc-a1f73dc37cb6 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:22.178566 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faad75d-f693-41fb-bdfc-a1f73dc37cb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:22.208570 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-9faad75d-f693-41fb-bdfc-a1f73dc37cb6 None None] MFA Rules not processed for user `144f9502176d4a18a578e9f4d1739ad0`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 20:09:22.209916 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-9faad75d-f693-41fb-bdfc-a1f73dc37cb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:09:22.219469 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:09:22.280263 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1427/2849] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 20:09:22 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 20:09:22.291712 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9b14ba-9969-42c2-95f2-c0a1039c9ba9 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:22.293418 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-ce9b14ba-9969-42c2-95f2-c0a1039c9ba9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:09:22.363859 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9b14ba-9969-42c2-95f2-c0a1039c9ba9 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:22.367464 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2993ad-b4ef-46a6-917c-5f74d8d5c7ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e79498873d4ed29a0e19bbea9b9214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:22.368018 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2993ad-b4ef-46a6-917c-5f74d8d5c7ed service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:22.368291 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2993ad-b4ef-46a6-917c-5f74d8d5c7ed service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:22.368558 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2993ad-b4ef-46a6-917c-5f74d8d5c7ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:22.377871 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2993ad-b4ef-46a6-917c-5f74d8d5c7ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:22.379271 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2993ad-b4ef-46a6-917c-5f74d8d5c7ed service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:22.388561 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1423/2850] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 20:09:22 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 20:09:22.453064 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faad75d-f693-41fb-bdfc-a1f73dc37cb6 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:22.513264 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faad75d-f693-41fb-bdfc-a1f73dc37cb6 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:22.517812 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b29189-3b60-4661-996e-baba4b5b5990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:22.518436 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b29189-3b60-4661-996e-baba4b5b5990 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:22.518713 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b29189-3b60-4661-996e-baba4b5b5990 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:22.518985 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b29189-3b60-4661-996e-baba4b5b5990 admin admin] PATH_INFO: `/v3/users/041f110de3cc4dc3b4f2850a6506e69d` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:22.520240 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b29189-3b60-4661-996e-baba4b5b5990 admin admin] RBAC: Authorizing `identity:delete_user(user_id=041f110de3cc4dc3b4f2850a6506e69d)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:22.522173 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b29189-3b60-4661-996e-baba4b5b5990 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:22.568088 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-90b29189-3b60-4661-996e-baba4b5b5990 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '041f110de3cc4dc3b4f2850a6506e69d'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:22.568755 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-90b29189-3b60-4661-996e-baba4b5b5990 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '041f110de3cc4dc3b4f2850a6506e69d'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:22.575851 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-90b29189-3b60-4661-996e-baba4b5b5990 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '041f110de3cc4dc3b4f2850a6506e69d'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:22.583115 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-90b29189-3b60-4661-996e-baba4b5b5990 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '041f110de3cc4dc3b4f2850a6506e69d'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:22.595454 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-90b29189-3b60-4661-996e-baba4b5b5990 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '041f110de3cc4dc3b4f2850a6506e69d'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:22.601342 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1428/2851] 10.222.0.61 () {58 vars in 1224 bytes} [Mon Aug 22 20:09:22 2022] DELETE /identity/v3/users/041f110de3cc4dc3b4f2850a6506e69d => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 20:09:22.605841 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56779e81-0c2e-43d6-9d8d-9003c9a1b03f None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:22.606170 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56779e81-0c2e-43d6-9d8d-9003c9a1b03f None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:22.606472 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56779e81-0c2e-43d6-9d8d-9003c9a1b03f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:22.625423 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-56779e81-0c2e-43d6-9d8d-9003c9a1b03f None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 20:09:22.626631 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-56779e81-0c2e-43d6-9d8d-9003c9a1b03f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:09:22.684064 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1424/2852] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 20:09:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 20:09:22.693341 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e294f0-578a-4fbb-9541-1f2d1013b506 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:22.693619 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e294f0-578a-4fbb-9541-1f2d1013b506 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:22.693890 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e294f0-578a-4fbb-9541-1f2d1013b506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:22.713622 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-90e294f0-578a-4fbb-9541-1f2d1013b506 None None] MFA Rules not processed for user `a4aca06c966a42f29f5c5bc108529d41`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 20:09:22.714825 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-90e294f0-578a-4fbb-9541-1f2d1013b506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:09:22.735473 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:09:22.767520 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1429/2853] 10.222.0.61 () {58 vars in 1062 bytes} [Mon Aug 22 20:09:22 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 20:09:22.773382 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56779e81-0c2e-43d6-9d8d-9003c9a1b03f None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:22.783208 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56779e81-0c2e-43d6-9d8d-9003c9a1b03f None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:22.786709 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49400d48-58f2-4100-bd67-aad386ffeecd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:22.787301 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49400d48-58f2-4100-bd67-aad386ffeecd service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:22.787573 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49400d48-58f2-4100-bd67-aad386ffeecd service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:22.787876 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49400d48-58f2-4100-bd67-aad386ffeecd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:22.797184 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49400d48-58f2-4100-bd67-aad386ffeecd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:22.798586 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49400d48-58f2-4100-bd67-aad386ffeecd service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:22.807919 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1425/2854] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 20:09:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 20:09:23.071421 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e294f0-578a-4fbb-9541-1f2d1013b506 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:23.073321 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-90e294f0-578a-4fbb-9541-1f2d1013b506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:09:23.087576 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e294f0-578a-4fbb-9541-1f2d1013b506 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:23.092225 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdecc59f-47bf-4a7b-8d84-89f4185953bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:23.092863 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdecc59f-47bf-4a7b-8d84-89f4185953bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:23.093137 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdecc59f-47bf-4a7b-8d84-89f4185953bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:23.093407 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdecc59f-47bf-4a7b-8d84-89f4185953bc admin admin] PATH_INFO: `/v3/projects/c857a712aed545ce97e8a74b893c0fa0` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:23.094544 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdecc59f-47bf-4a7b-8d84-89f4185953bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=c857a712aed545ce97e8a74b893c0fa0)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:23.096904 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdecc59f-47bf-4a7b-8d84-89f4185953bc admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:23.145149 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-cdecc59f-47bf-4a7b-8d84-89f4185953bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c857a712aed545ce97e8a74b893c0fa0', 'request_id': 'req-cdecc59f-47bf-4a7b-8d84-89f4185953bc'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:23.152471 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-cdecc59f-47bf-4a7b-8d84-89f4185953bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c857a712aed545ce97e8a74b893c0fa0', 'request_id': 'req-cdecc59f-47bf-4a7b-8d84-89f4185953bc'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:23.158713 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-cdecc59f-47bf-4a7b-8d84-89f4185953bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c857a712aed545ce97e8a74b893c0fa0', 'request_id': 'req-cdecc59f-47bf-4a7b-8d84-89f4185953bc'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:23.169476 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-cdecc59f-47bf-4a7b-8d84-89f4185953bc admin admin] The token cache is being invalidate because project c857a712aed545ce97e8a74b893c0fa0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 20:09:23.169925 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-cdecc59f-47bf-4a7b-8d84-89f4185953bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:23.170658 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-cdecc59f-47bf-4a7b-8d84-89f4185953bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:23.172990 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1430/2855] 10.222.0.61 () {58 vars in 1233 bytes} [Mon Aug 22 20:09:23 2022] DELETE /identity/v3/projects/c857a712aed545ce97e8a74b893c0fa0 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 20:09:23.176978 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49400d48-58f2-4100-bd67-aad386ffeecd service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:23.186499 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49400d48-58f2-4100-bd67-aad386ffeecd service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:23.210835 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:09:23.243473 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebeef50-adeb-4a25-9ae8-5a2800dfffb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:23.244123 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebeef50-adeb-4a25-9ae8-5a2800dfffb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:23.244398 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebeef50-adeb-4a25-9ae8-5a2800dfffb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:23.244665 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebeef50-adeb-4a25-9ae8-5a2800dfffb1 admin admin] PATH_INFO: `/v3/users/144f9502176d4a18a578e9f4d1739ad0` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:23.245770 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebeef50-adeb-4a25-9ae8-5a2800dfffb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=144f9502176d4a18a578e9f4d1739ad0)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:23.247669 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebeef50-adeb-4a25-9ae8-5a2800dfffb1 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:23.294093 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-bebeef50-adeb-4a25-9ae8-5a2800dfffb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '144f9502176d4a18a578e9f4d1739ad0'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:23.294751 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-bebeef50-adeb-4a25-9ae8-5a2800dfffb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '144f9502176d4a18a578e9f4d1739ad0'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:23.302271 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-bebeef50-adeb-4a25-9ae8-5a2800dfffb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '144f9502176d4a18a578e9f4d1739ad0'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:23.309846 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-bebeef50-adeb-4a25-9ae8-5a2800dfffb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '144f9502176d4a18a578e9f4d1739ad0'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:23.321699 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-bebeef50-adeb-4a25-9ae8-5a2800dfffb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '144f9502176d4a18a578e9f4d1739ad0'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:23.328095 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1426/2856] 10.222.0.61 () {58 vars in 1224 bytes} [Mon Aug 22 20:09:23 2022] DELETE /identity/v3/users/144f9502176d4a18a578e9f4d1739ad0 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 20:09:23.604154 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdecc59f-47bf-4a7b-8d84-89f4185953bc admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:23.605585 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-cdecc59f-47bf-4a7b-8d84-89f4185953bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:09:23.619127 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdecc59f-47bf-4a7b-8d84-89f4185953bc admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:23.677930 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4c7dc6-3908-4fee-aad7-0dd5215d071c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:23.678593 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4c7dc6-3908-4fee-aad7-0dd5215d071c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:23.678868 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4c7dc6-3908-4fee-aad7-0dd5215d071c admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:23.679137 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4c7dc6-3908-4fee-aad7-0dd5215d071c admin admin] PATH_INFO: `/v3/projects/82962266040f437a9a0c5f6132dad0d9` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:23.680341 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4c7dc6-3908-4fee-aad7-0dd5215d071c admin admin] RBAC: Authorizing `identity:delete_project(project_id=82962266040f437a9a0c5f6132dad0d9)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:23.682309 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4c7dc6-3908-4fee-aad7-0dd5215d071c admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:23.728162 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-6b4c7dc6-3908-4fee-aad7-0dd5215d071c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82962266040f437a9a0c5f6132dad0d9', 'request_id': 'req-6b4c7dc6-3908-4fee-aad7-0dd5215d071c'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:23.735295 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-6b4c7dc6-3908-4fee-aad7-0dd5215d071c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82962266040f437a9a0c5f6132dad0d9', 'request_id': 'req-6b4c7dc6-3908-4fee-aad7-0dd5215d071c'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:23.741498 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-6b4c7dc6-3908-4fee-aad7-0dd5215d071c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82962266040f437a9a0c5f6132dad0d9', 'request_id': 'req-6b4c7dc6-3908-4fee-aad7-0dd5215d071c'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:23.742713 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:09:23.754178 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-6b4c7dc6-3908-4fee-aad7-0dd5215d071c admin admin] The token cache is being invalidate because project 82962266040f437a9a0c5f6132dad0d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 20:09:23.754488 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-6b4c7dc6-3908-4fee-aad7-0dd5215d071c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:23.755133 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-6b4c7dc6-3908-4fee-aad7-0dd5215d071c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:23.756614 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1431/2857] 10.222.0.61 () {58 vars in 1233 bytes} [Mon Aug 22 20:09:23 2022] DELETE /identity/v3/projects/82962266040f437a9a0c5f6132dad0d9 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 20:09:23.761646 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2836b1fa-d7ee-45f4-ad9a-56f1be6e7389 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:23.761928 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2836b1fa-d7ee-45f4-ad9a-56f1be6e7389 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:23.762198 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2836b1fa-d7ee-45f4-ad9a-56f1be6e7389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:23.782308 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-2836b1fa-d7ee-45f4-ad9a-56f1be6e7389 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 20:09:23.783502 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-2836b1fa-d7ee-45f4-ad9a-56f1be6e7389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:09:23.841587 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1427/2858] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 20:09:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 20:09:23.849683 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d23cfa-451a-4022-87f4-d99826ef2c65 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:23.849968 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d23cfa-451a-4022-87f4-d99826ef2c65 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:23.850244 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d23cfa-451a-4022-87f4-d99826ef2c65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:23.869134 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-f3d23cfa-451a-4022-87f4-d99826ef2c65 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 20:09:23.870326 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-f3d23cfa-451a-4022-87f4-d99826ef2c65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:09:23.878305 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1432/2859] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 20:09:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 20:09:23.882345 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2836b1fa-d7ee-45f4-ad9a-56f1be6e7389 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:23.892559 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2836b1fa-d7ee-45f4-ad9a-56f1be6e7389 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:23.897146 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a8bd73-cda7-4f10-bc20-1563f92a2062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:23.897719 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a8bd73-cda7-4f10-bc20-1563f92a2062 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:23.897994 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a8bd73-cda7-4f10-bc20-1563f92a2062 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:23.898262 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a8bd73-cda7-4f10-bc20-1563f92a2062 admin admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:23.898903 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a8bd73-cda7-4f10-bc20-1563f92a2062 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 20:09:23.899200 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a8bd73-cda7-4f10-bc20-1563f92a2062 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:23.900817 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a8bd73-cda7-4f10-bc20-1563f92a2062 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:23.912551 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1428/2860] 10.222.0.61 () {58 vars in 1153 bytes} [Mon Aug 22 20:09:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 20:09:23.916813 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d23cfa-451a-4022-87f4-d99826ef2c65 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:23.926018 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d23cfa-451a-4022-87f4-d99826ef2c65 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:23.930652 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314082ee-b87d-40dc-b9e5-e7f715640ed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:23.931184 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314082ee-b87d-40dc-b9e5-e7f715640ed5 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:23.931458 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314082ee-b87d-40dc-b9e5-e7f715640ed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:23.931755 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314082ee-b87d-40dc-b9e5-e7f715640ed5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:23.932381 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314082ee-b87d-40dc-b9e5-e7f715640ed5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:23.934251 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314082ee-b87d-40dc-b9e5-e7f715640ed5 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:23.961773 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1433/2861] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 20:09:23 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 20:09:23.965953 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a8bd73-cda7-4f10-bc20-1563f92a2062 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:23.976621 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a8bd73-cda7-4f10-bc20-1563f92a2062 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:24.006093 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:09:24.034272 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6558828-0144-4c72-a003-2181dfe59f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:24.034881 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6558828-0144-4c72-a003-2181dfe59f85 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:24.035159 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6558828-0144-4c72-a003-2181dfe59f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:24.035430 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6558828-0144-4c72-a003-2181dfe59f85 admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:24.036186 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6558828-0144-4c72-a003-2181dfe59f85 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:24.038088 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6558828-0144-4c72-a003-2181dfe59f85 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:24.057971 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1429/2862] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 20:09:23 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 20:09:24.062727 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314082ee-b87d-40dc-b9e5-e7f715640ed5 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:24.073209 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314082ee-b87d-40dc-b9e5-e7f715640ed5 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:24.078438 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38c352a-28cc-453b-b3b7-05f5898e5fa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:24.079025 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38c352a-28cc-453b-b3b7-05f5898e5fa6 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:24.079297 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38c352a-28cc-453b-b3b7-05f5898e5fa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:24.079566 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38c352a-28cc-453b-b3b7-05f5898e5fa6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:24.080231 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38c352a-28cc-453b-b3b7-05f5898e5fa6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:24.081807 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38c352a-28cc-453b-b3b7-05f5898e5fa6 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:24.091401 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1434/2863] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 20:09:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 20:09:24.095336 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6558828-0144-4c72-a003-2181dfe59f85 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:24.105227 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6558828-0144-4c72-a003-2181dfe59f85 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:24.110029 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6fe450-6cb9-4920-a846-11471be05493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:24.110631 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6fe450-6cb9-4920-a846-11471be05493 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:24.110904 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6fe450-6cb9-4920-a846-11471be05493 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:24.111170 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6fe450-6cb9-4920-a846-11471be05493 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:24.111858 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6fe450-6cb9-4920-a846-11471be05493 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:24.113462 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6fe450-6cb9-4920-a846-11471be05493 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:24.124245 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1430/2864] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 20:09:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 20:09:24.128824 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38c352a-28cc-453b-b3b7-05f5898e5fa6 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:24.140755 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38c352a-28cc-453b-b3b7-05f5898e5fa6 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:24.145466 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ba7850-0c7d-4389-bc26-c1b2000087b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:24.146126 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ba7850-0c7d-4389-bc26-c1b2000087b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:24.146402 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ba7850-0c7d-4389-bc26-c1b2000087b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:24.147035 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ba7850-0c7d-4389-bc26-c1b2000087b0 admin admin] PATH_INFO: `/v3/projects/0726cdcabbe34ba7bc5814a31fdda79f/users/e19d584c582d457daf1547ef3247bddf/roles/74a40bc819cb42eea7e966a4b2a016f1` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:24.161313 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ba7850-0c7d-4389-bc26-c1b2000087b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0726cdcabbe34ba7bc5814a31fdda79f, user_id=e19d584c582d457daf1547ef3247bddf, role_id=74a40bc819cb42eea7e966a4b2a016f1)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:24.163958 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ba7850-0c7d-4389-bc26-c1b2000087b0 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:24.176226 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1435/2865] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 20:09:24 2022] PUT /identity/v3/projects/0726cdcabbe34ba7bc5814a31fdda79f/users/e19d584c582d457daf1547ef3247bddf/roles/74a40bc819cb42eea7e966a4b2a016f1 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 20:09:24.192299 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9d9117-da0b-4393-864e-ac4ac01f58d7 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:24.193049 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9d9117-da0b-4393-864e-ac4ac01f58d7 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:24.193049 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9d9117-da0b-4393-864e-ac4ac01f58d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:24.224132 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-ab9d9117-da0b-4393-864e-ac4ac01f58d7 None None] MFA Rules not processed for user `e19d584c582d457daf1547ef3247bddf`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 20:09:24.225360 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-ab9d9117-da0b-4393-864e-ac4ac01f58d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:09:24.288321 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1431/2866] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 20:09:24 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 20:09:24.292796 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ba7850-0c7d-4389-bc26-c1b2000087b0 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:24.353035 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ba7850-0c7d-4389-bc26-c1b2000087b0 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:24.357592 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3500a278-5e6f-4f4e-93ba-f39278328129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:24.358188 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3500a278-5e6f-4f4e-93ba-f39278328129 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:24.358464 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3500a278-5e6f-4f4e-93ba-f39278328129 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:24.358759 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3500a278-5e6f-4f4e-93ba-f39278328129 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:24.359393 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3500a278-5e6f-4f4e-93ba-f39278328129 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:24.361309 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3500a278-5e6f-4f4e-93ba-f39278328129 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:24.386844 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1436/2867] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 20:09:24 2022] POST /identity/v3/projects => generated 381 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 20:09:24.391450 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9d9117-da0b-4393-864e-ac4ac01f58d7 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:24.402307 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9d9117-da0b-4393-864e-ac4ac01f58d7 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:24.459505 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eccea40-ce2e-41d6-b5b7-f2541edf3f9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:24.460064 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eccea40-ce2e-41d6-b5b7-f2541edf3f9d admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:24.460340 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eccea40-ce2e-41d6-b5b7-f2541edf3f9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:24.460608 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eccea40-ce2e-41d6-b5b7-f2541edf3f9d admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:24.461226 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eccea40-ce2e-41d6-b5b7-f2541edf3f9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:24.463109 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eccea40-ce2e-41d6-b5b7-f2541edf3f9d admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:24.482873 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1432/2868] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 20:09:24 2022] POST /identity/v3/users => generated 297 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 20:09:24.487500 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3500a278-5e6f-4f4e-93ba-f39278328129 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:24.498202 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3500a278-5e6f-4f4e-93ba-f39278328129 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:24.503460 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a0f986-ddd1-41f9-991f-0e74750bc7e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:24.504094 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a0f986-ddd1-41f9-991f-0e74750bc7e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:24.504373 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a0f986-ddd1-41f9-991f-0e74750bc7e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:24.504649 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a0f986-ddd1-41f9-991f-0e74750bc7e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:24.505286 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a0f986-ddd1-41f9-991f-0e74750bc7e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:24.506865 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a0f986-ddd1-41f9-991f-0e74750bc7e6 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:24.517223 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1437/2869] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 20:09:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 20:09:24.521237 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eccea40-ce2e-41d6-b5b7-f2541edf3f9d admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:24.531394 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eccea40-ce2e-41d6-b5b7-f2541edf3f9d admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:24.536090 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62c1361-0b1e-43dc-b4f8-6046081e60cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:24.536758 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62c1361-0b1e-43dc-b4f8-6046081e60cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:24.537034 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62c1361-0b1e-43dc-b4f8-6046081e60cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:24.537303 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62c1361-0b1e-43dc-b4f8-6046081e60cd admin admin] PATH_INFO: `/v3/projects/0bd71b37359a48db88ba063c4f915401/users/7c0a27b3102948d3afb0191fca0c4502/roles/163c30c3b3c246beb849ce176f02d9da` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:24.552166 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62c1361-0b1e-43dc-b4f8-6046081e60cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=0bd71b37359a48db88ba063c4f915401, user_id=7c0a27b3102948d3afb0191fca0c4502, role_id=163c30c3b3c246beb849ce176f02d9da)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:24.554795 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62c1361-0b1e-43dc-b4f8-6046081e60cd admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:24.567696 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1433/2870] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 20:09:24 2022] PUT /identity/v3/projects/0bd71b37359a48db88ba063c4f915401/users/7c0a27b3102948d3afb0191fca0c4502/roles/163c30c3b3c246beb849ce176f02d9da => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 20:09:24.571569 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a0f986-ddd1-41f9-991f-0e74750bc7e6 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:24.581813 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a0f986-ddd1-41f9-991f-0e74750bc7e6 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:24.637116 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e6a231-fd4e-42ca-aa02-146dd3f1494b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:24.637720 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e6a231-fd4e-42ca-aa02-146dd3f1494b admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:24.638053 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e6a231-fd4e-42ca-aa02-146dd3f1494b admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:24.638367 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e6a231-fd4e-42ca-aa02-146dd3f1494b admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:24.639099 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e6a231-fd4e-42ca-aa02-146dd3f1494b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:24.640749 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e6a231-fd4e-42ca-aa02-146dd3f1494b admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:24.641963 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:09:24.655117 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1438/2871] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 20:09:24 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 20:09:24.656390 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62c1361-0b1e-43dc-b4f8-6046081e60cd admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:24.666701 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62c1361-0b1e-43dc-b4f8-6046081e60cd admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:24.671842 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770ab39a-4db2-47db-952c-7c875cf3f06c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:24.672485 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770ab39a-4db2-47db-952c-7c875cf3f06c admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:24.672762 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770ab39a-4db2-47db-952c-7c875cf3f06c admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:24.673034 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770ab39a-4db2-47db-952c-7c875cf3f06c admin admin] PATH_INFO: `/v3/projects/0bd71b37359a48db88ba063c4f915401/users/7c0a27b3102948d3afb0191fca0c4502/roles/163c30c3b3c246beb849ce176f02d9da` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:24.674951 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770ab39a-4db2-47db-952c-7c875cf3f06c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0bd71b37359a48db88ba063c4f915401, user_id=7c0a27b3102948d3afb0191fca0c4502, role_id=163c30c3b3c246beb849ce176f02d9da)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:24.677591 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770ab39a-4db2-47db-952c-7c875cf3f06c admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:24.679794 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:09:24.689762 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1434/2872] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 20:09:24 2022] PUT /identity/v3/projects/0bd71b37359a48db88ba063c4f915401/users/7c0a27b3102948d3afb0191fca0c4502/roles/163c30c3b3c246beb849ce176f02d9da => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 20:09:24.700267 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0736de3b-dd98-46cf-bef9-d701c1fea85a None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:24.700808 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0736de3b-dd98-46cf-bef9-d701c1fea85a None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:24.701193 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0736de3b-dd98-46cf-bef9-d701c1fea85a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:24.729977 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-0736de3b-dd98-46cf-bef9-d701c1fea85a None None] MFA Rules not processed for user `7c0a27b3102948d3afb0191fca0c4502`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 20:09:24.731172 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-0736de3b-dd98-46cf-bef9-d701c1fea85a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:09:24.796975 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1439/2873] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 20:09:24 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 20:09:24.807625 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770ab39a-4db2-47db-952c-7c875cf3f06c admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:24.809424 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-770ab39a-4db2-47db-952c-7c875cf3f06c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:09:24.866337 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770ab39a-4db2-47db-952c-7c875cf3f06c admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:24.870001 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b18235-06d9-4dd6-94fd-1caf7b609649 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e79498873d4ed29a0e19bbea9b9214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:24.870591 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b18235-06d9-4dd6-94fd-1caf7b609649 service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:24.870866 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b18235-06d9-4dd6-94fd-1caf7b609649 service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:24.871133 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b18235-06d9-4dd6-94fd-1caf7b609649 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:24.880735 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b18235-06d9-4dd6-94fd-1caf7b609649 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:24.882164 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b18235-06d9-4dd6-94fd-1caf7b609649 service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:24.892304 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1435/2874] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 20:09:24 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 20:09:24.990878 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0736de3b-dd98-46cf-bef9-d701c1fea85a None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:25.001844 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0736de3b-dd98-46cf-bef9-d701c1fea85a None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:25.005430 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c55bb52-b7f8-4834-8006-ae29ffe318a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e79498873d4ed29a0e19bbea9b9214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:25.006033 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c55bb52-b7f8-4834-8006-ae29ffe318a8 service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:25.006311 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c55bb52-b7f8-4834-8006-ae29ffe318a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:25.006577 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c55bb52-b7f8-4834-8006-ae29ffe318a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:25.060643 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c55bb52-b7f8-4834-8006-ae29ffe318a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:25.062575 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c55bb52-b7f8-4834-8006-ae29ffe318a8 service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:25.071210 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1440/2875] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 20:09:24 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 20:09:27.648154 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b18235-06d9-4dd6-94fd-1caf7b609649 service cinder] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:27.710039 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b18235-06d9-4dd6-94fd-1caf7b609649 service cinder] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:27.714511 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca76005-c635-4688-b8f6-f3b13025baa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:27.715134 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca76005-c635-4688-b8f6-f3b13025baa3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:27.715410 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca76005-c635-4688-b8f6-f3b13025baa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:27.715718 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca76005-c635-4688-b8f6-f3b13025baa3 admin admin] PATH_INFO: `/v3/users/e19d584c582d457daf1547ef3247bddf` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:27.716943 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca76005-c635-4688-b8f6-f3b13025baa3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e19d584c582d457daf1547ef3247bddf)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:27.718841 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca76005-c635-4688-b8f6-f3b13025baa3 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:27.764939 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-cca76005-c635-4688-b8f6-f3b13025baa3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e19d584c582d457daf1547ef3247bddf'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:27.765578 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-cca76005-c635-4688-b8f6-f3b13025baa3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e19d584c582d457daf1547ef3247bddf'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:27.772492 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-cca76005-c635-4688-b8f6-f3b13025baa3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e19d584c582d457daf1547ef3247bddf'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:27.779624 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-cca76005-c635-4688-b8f6-f3b13025baa3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e19d584c582d457daf1547ef3247bddf'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:27.792813 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-cca76005-c635-4688-b8f6-f3b13025baa3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e19d584c582d457daf1547ef3247bddf'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:27.799581 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1436/2876] 10.222.0.61 () {58 vars in 1224 bytes} [Mon Aug 22 20:09:27 2022] DELETE /identity/v3/users/e19d584c582d457daf1547ef3247bddf => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 20:09:27.804091 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84bc67f-3cbb-487c-a9a6-371dad6fbd22 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:27.804578 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84bc67f-3cbb-487c-a9a6-371dad6fbd22 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:27.804673 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84bc67f-3cbb-487c-a9a6-371dad6fbd22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:27.823667 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-a84bc67f-3cbb-487c-a9a6-371dad6fbd22 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 20:09:27.824868 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-a84bc67f-3cbb-487c-a9a6-371dad6fbd22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:09:27.883023 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1441/2877] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 20:09:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 20:09:27.892502 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca76005-c635-4688-b8f6-f3b13025baa3 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:27.893898 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-cca76005-c635-4688-b8f6-f3b13025baa3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:09:27.912628 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:09:27.952732 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca76005-c635-4688-b8f6-f3b13025baa3 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:27.956425 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bbfe3e-81d0-4ce0-8757-8deeee1d0d0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:27.957024 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bbfe3e-81d0-4ce0-8757-8deeee1d0d0c service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:27.957304 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bbfe3e-81d0-4ce0-8757-8deeee1d0d0c service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:27.957577 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bbfe3e-81d0-4ce0-8757-8deeee1d0d0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:27.966460 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bbfe3e-81d0-4ce0-8757-8deeee1d0d0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:27.967901 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bbfe3e-81d0-4ce0-8757-8deeee1d0d0c service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:27.978051 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1437/2878] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 20:09:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 20:09:28.244573 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84bc67f-3cbb-487c-a9a6-371dad6fbd22 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:28.245993 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-a84bc67f-3cbb-487c-a9a6-371dad6fbd22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:09:28.260515 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84bc67f-3cbb-487c-a9a6-371dad6fbd22 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:28.265796 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dc0ae0-58b3-43b1-a175-37ca3b5e27b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:28.266618 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dc0ae0-58b3-43b1-a175-37ca3b5e27b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:28.267033 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dc0ae0-58b3-43b1-a175-37ca3b5e27b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:28.267172 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dc0ae0-58b3-43b1-a175-37ca3b5e27b1 admin admin] PATH_INFO: `/v3/projects/0726cdcabbe34ba7bc5814a31fdda79f` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:28.268365 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dc0ae0-58b3-43b1-a175-37ca3b5e27b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0726cdcabbe34ba7bc5814a31fdda79f)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:28.270308 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dc0ae0-58b3-43b1-a175-37ca3b5e27b1 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:28.286524 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:09:28.317256 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-15dc0ae0-58b3-43b1-a175-37ca3b5e27b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0726cdcabbe34ba7bc5814a31fdda79f', 'request_id': 'req-15dc0ae0-58b3-43b1-a175-37ca3b5e27b1'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:28.324482 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-15dc0ae0-58b3-43b1-a175-37ca3b5e27b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0726cdcabbe34ba7bc5814a31fdda79f', 'request_id': 'req-15dc0ae0-58b3-43b1-a175-37ca3b5e27b1'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:28.331057 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-15dc0ae0-58b3-43b1-a175-37ca3b5e27b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0726cdcabbe34ba7bc5814a31fdda79f', 'request_id': 'req-15dc0ae0-58b3-43b1-a175-37ca3b5e27b1'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:28.341262 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-15dc0ae0-58b3-43b1-a175-37ca3b5e27b1 admin admin] The token cache is being invalidate because project 0726cdcabbe34ba7bc5814a31fdda79f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 20:09:28.341546 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-15dc0ae0-58b3-43b1-a175-37ca3b5e27b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:28.342159 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-15dc0ae0-58b3-43b1-a175-37ca3b5e27b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:28.344267 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1442/2879] 10.222.0.61 () {58 vars in 1233 bytes} [Mon Aug 22 20:09:28 2022] DELETE /identity/v3/projects/0726cdcabbe34ba7bc5814a31fdda79f => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 20:09:28.348361 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bbfe3e-81d0-4ce0-8757-8deeee1d0d0c service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:28.358181 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bbfe3e-81d0-4ce0-8757-8deeee1d0d0c service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:28.414462 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e1b4ea-5de1-4014-933e-135756c3cc76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:28.415011 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e1b4ea-5de1-4014-933e-135756c3cc76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:28.415289 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e1b4ea-5de1-4014-933e-135756c3cc76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:28.415559 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e1b4ea-5de1-4014-933e-135756c3cc76 admin admin] PATH_INFO: `/v3/users/7c0a27b3102948d3afb0191fca0c4502` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:28.416712 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e1b4ea-5de1-4014-933e-135756c3cc76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c0a27b3102948d3afb0191fca0c4502)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:28.418641 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e1b4ea-5de1-4014-933e-135756c3cc76 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:28.463446 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a5e1b4ea-5de1-4014-933e-135756c3cc76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c0a27b3102948d3afb0191fca0c4502'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:28.464103 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a5e1b4ea-5de1-4014-933e-135756c3cc76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c0a27b3102948d3afb0191fca0c4502'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:28.471370 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a5e1b4ea-5de1-4014-933e-135756c3cc76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c0a27b3102948d3afb0191fca0c4502'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:28.478662 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a5e1b4ea-5de1-4014-933e-135756c3cc76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c0a27b3102948d3afb0191fca0c4502'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:28.490204 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a5e1b4ea-5de1-4014-933e-135756c3cc76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c0a27b3102948d3afb0191fca0c4502'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:28.495904 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1438/2880] 10.222.0.61 () {58 vars in 1224 bytes} [Mon Aug 22 20:09:28 2022] DELETE /identity/v3/users/7c0a27b3102948d3afb0191fca0c4502 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 20:09:28.763073 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dc0ae0-58b3-43b1-a175-37ca3b5e27b1 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:28.764537 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-15dc0ae0-58b3-43b1-a175-37ca3b5e27b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:09:28.778243 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dc0ae0-58b3-43b1-a175-37ca3b5e27b1 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:28.835554 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0d932c-90b4-4756-9989-c156003f8e79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:28.836269 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0d932c-90b4-4756-9989-c156003f8e79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:28.836491 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0d932c-90b4-4756-9989-c156003f8e79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:28.836756 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0d932c-90b4-4756-9989-c156003f8e79 admin admin] PATH_INFO: `/v3/projects/0bd71b37359a48db88ba063c4f915401` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:28.837863 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0d932c-90b4-4756-9989-c156003f8e79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0bd71b37359a48db88ba063c4f915401)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:28.839759 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0d932c-90b4-4756-9989-c156003f8e79 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:28.883738 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-4f0d932c-90b4-4756-9989-c156003f8e79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0bd71b37359a48db88ba063c4f915401', 'request_id': 'req-4f0d932c-90b4-4756-9989-c156003f8e79'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:28.891266 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-4f0d932c-90b4-4756-9989-c156003f8e79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0bd71b37359a48db88ba063c4f915401', 'request_id': 'req-4f0d932c-90b4-4756-9989-c156003f8e79'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:28.897358 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-4f0d932c-90b4-4756-9989-c156003f8e79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0bd71b37359a48db88ba063c4f915401', 'request_id': 'req-4f0d932c-90b4-4756-9989-c156003f8e79'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:28.907303 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-4f0d932c-90b4-4756-9989-c156003f8e79 admin admin] The token cache is being invalidate because project 0bd71b37359a48db88ba063c4f915401 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 20:09:28.907584 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-4f0d932c-90b4-4756-9989-c156003f8e79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:28.908220 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-4f0d932c-90b4-4756-9989-c156003f8e79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:28.909656 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1443/2881] 10.222.0.61 () {58 vars in 1233 bytes} [Mon Aug 22 20:09:28 2022] DELETE /identity/v3/projects/0bd71b37359a48db88ba063c4f915401 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 20:09:28.914620 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f148b8-9c84-4222-8a7b-3695ca85f876 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:28.915107 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f148b8-9c84-4222-8a7b-3695ca85f876 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:28.915225 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f148b8-9c84-4222-8a7b-3695ca85f876 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:28.935048 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-42f148b8-9c84-4222-8a7b-3695ca85f876 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 20:09:28.936227 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-42f148b8-9c84-4222-8a7b-3695ca85f876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:09:28.968378 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:09:28.996699 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1439/2882] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 20:09:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 20:09:29.005904 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd301bf7-97ea-46e5-9da0-28b9d637b314 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:29.006199 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd301bf7-97ea-46e5-9da0-28b9d637b314 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:29.006502 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd301bf7-97ea-46e5-9da0-28b9d637b314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:29.025933 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-fd301bf7-97ea-46e5-9da0-28b9d637b314 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 20:09:29.027157 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-fd301bf7-97ea-46e5-9da0-28b9d637b314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:09:29.035191 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1444/2883] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 20:09:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 20:09:29.040769 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f148b8-9c84-4222-8a7b-3695ca85f876 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:29.051015 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f148b8-9c84-4222-8a7b-3695ca85f876 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:29.055525 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8f8c45-8926-46b8-901a-dad521e1ef58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:29.056154 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8f8c45-8926-46b8-901a-dad521e1ef58 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:29.056434 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8f8c45-8926-46b8-901a-dad521e1ef58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:29.056711 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8f8c45-8926-46b8-901a-dad521e1ef58 admin admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:29.057364 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8f8c45-8926-46b8-901a-dad521e1ef58 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 20:09:29.057666 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8f8c45-8926-46b8-901a-dad521e1ef58 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:29.059252 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8f8c45-8926-46b8-901a-dad521e1ef58 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:29.071169 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1440/2884] 10.222.0.61 () {58 vars in 1153 bytes} [Mon Aug 22 20:09:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 20:09:29.075505 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd301bf7-97ea-46e5-9da0-28b9d637b314 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:29.084676 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd301bf7-97ea-46e5-9da0-28b9d637b314 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:29.089242 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d650614c-c2aa-4f3a-b06a-23c324f17702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:29.089826 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d650614c-c2aa-4f3a-b06a-23c324f17702 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:29.090104 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d650614c-c2aa-4f3a-b06a-23c324f17702 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:29.090369 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d650614c-c2aa-4f3a-b06a-23c324f17702 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:29.090989 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d650614c-c2aa-4f3a-b06a-23c324f17702 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:29.092983 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d650614c-c2aa-4f3a-b06a-23c324f17702 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:29.121968 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1445/2885] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 20:09:29 2022] POST /identity/v3/projects => generated 387 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 20:09:29.126923 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8f8c45-8926-46b8-901a-dad521e1ef58 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:29.137123 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8f8c45-8926-46b8-901a-dad521e1ef58 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:29.194290 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c216980b-c4ab-4b69-bb16-e91ebc31ed2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:29.194844 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c216980b-c4ab-4b69-bb16-e91ebc31ed2f admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:29.195125 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c216980b-c4ab-4b69-bb16-e91ebc31ed2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:29.195398 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c216980b-c4ab-4b69-bb16-e91ebc31ed2f admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:29.196055 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c216980b-c4ab-4b69-bb16-e91ebc31ed2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:29.198203 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c216980b-c4ab-4b69-bb16-e91ebc31ed2f admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:29.218830 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1441/2886] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 20:09:29 2022] POST /identity/v3/users => generated 294 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 20:09:29.223009 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d650614c-c2aa-4f3a-b06a-23c324f17702 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:29.229635 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:09:29.234150 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d650614c-c2aa-4f3a-b06a-23c324f17702 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:29.239950 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed657e9-1106-4ce5-9ae0-b49a9a9652a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:29.240705 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed657e9-1106-4ce5-9ae0-b49a9a9652a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:29.241162 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed657e9-1106-4ce5-9ae0-b49a9a9652a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:29.241615 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed657e9-1106-4ce5-9ae0-b49a9a9652a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:29.242250 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed657e9-1106-4ce5-9ae0-b49a9a9652a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:29.243870 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed657e9-1106-4ce5-9ae0-b49a9a9652a6 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:29.254236 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1446/2887] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 20:09:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 20:09:29.258172 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c216980b-c4ab-4b69-bb16-e91ebc31ed2f admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:29.268105 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c216980b-c4ab-4b69-bb16-e91ebc31ed2f admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:29.272761 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cbe76e-794d-4485-9a35-e9f1b4cce569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:29.273346 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cbe76e-794d-4485-9a35-e9f1b4cce569 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:29.273617 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cbe76e-794d-4485-9a35-e9f1b4cce569 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:29.273886 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cbe76e-794d-4485-9a35-e9f1b4cce569 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:29.274543 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cbe76e-794d-4485-9a35-e9f1b4cce569 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:29.276152 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cbe76e-794d-4485-9a35-e9f1b4cce569 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:29.288001 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1442/2888] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 20:09:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 20:09:29.291911 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed657e9-1106-4ce5-9ae0-b49a9a9652a6 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:29.302411 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed657e9-1106-4ce5-9ae0-b49a9a9652a6 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:29.307059 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc5c021-32f0-44f8-8ca0-853dc3df33e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:29.307743 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc5c021-32f0-44f8-8ca0-853dc3df33e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:29.308050 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc5c021-32f0-44f8-8ca0-853dc3df33e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:29.308327 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc5c021-32f0-44f8-8ca0-853dc3df33e4 admin admin] PATH_INFO: `/v3/projects/a91f33e61df34a95a72c6e218d3b66f7/users/1d42584c8731425487c20bd4a2cbfc14/roles/74a40bc819cb42eea7e966a4b2a016f1` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:29.322742 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc5c021-32f0-44f8-8ca0-853dc3df33e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a91f33e61df34a95a72c6e218d3b66f7, user_id=1d42584c8731425487c20bd4a2cbfc14, role_id=74a40bc819cb42eea7e966a4b2a016f1)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:29.325413 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc5c021-32f0-44f8-8ca0-853dc3df33e4 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:29.338940 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1447/2889] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 20:09:29 2022] PUT /identity/v3/projects/a91f33e61df34a95a72c6e218d3b66f7/users/1d42584c8731425487c20bd4a2cbfc14/roles/74a40bc819cb42eea7e966a4b2a016f1 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 20:09:29.349543 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2153cded-8128-4c3a-ac11-082e1a2a3cc1 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:29.349829 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2153cded-8128-4c3a-ac11-082e1a2a3cc1 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:29.350101 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2153cded-8128-4c3a-ac11-082e1a2a3cc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:29.377499 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-2153cded-8128-4c3a-ac11-082e1a2a3cc1 None None] MFA Rules not processed for user `1d42584c8731425487c20bd4a2cbfc14`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 20:09:29.378711 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-2153cded-8128-4c3a-ac11-082e1a2a3cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:09:29.442776 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1443/2890] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 20:09:29 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 20:09:29.447717 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc5c021-32f0-44f8-8ca0-853dc3df33e4 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:29.511224 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc5c021-32f0-44f8-8ca0-853dc3df33e4 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:29.515936 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b56395-5142-4456-97ad-f205cb32de8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:29.516542 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b56395-5142-4456-97ad-f205cb32de8d admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:29.516820 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b56395-5142-4456-97ad-f205cb32de8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:29.517090 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b56395-5142-4456-97ad-f205cb32de8d admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:29.517711 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b56395-5142-4456-97ad-f205cb32de8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:29.519708 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b56395-5142-4456-97ad-f205cb32de8d admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:29.546160 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1448/2891] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 20:09:29 2022] POST /identity/v3/projects => generated 387 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 20:09:29.550675 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2153cded-8128-4c3a-ac11-082e1a2a3cc1 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:29.561074 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2153cded-8128-4c3a-ac11-082e1a2a3cc1 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:29.589781 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:09:29.618538 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b39ecb-a35b-4105-a523-8ab7de9b53b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:29.619146 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b39ecb-a35b-4105-a523-8ab7de9b53b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:29.619456 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b39ecb-a35b-4105-a523-8ab7de9b53b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:29.619758 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b39ecb-a35b-4105-a523-8ab7de9b53b9 admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:29.620395 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b39ecb-a35b-4105-a523-8ab7de9b53b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:29.622297 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b39ecb-a35b-4105-a523-8ab7de9b53b9 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:29.643565 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1444/2892] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 20:09:29 2022] POST /identity/v3/users => generated 300 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 20:09:29.647983 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b56395-5142-4456-97ad-f205cb32de8d admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:29.659149 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b56395-5142-4456-97ad-f205cb32de8d admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:29.663818 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eeeb6f-6db0-4bfe-a7c1-feb94f2d43e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:29.664408 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eeeb6f-6db0-4bfe-a7c1-feb94f2d43e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:29.664686 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eeeb6f-6db0-4bfe-a7c1-feb94f2d43e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:29.664961 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eeeb6f-6db0-4bfe-a7c1-feb94f2d43e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:29.665598 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58eeeb6f-6db0-4bfe-a7c1-feb94f2d43e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:29.667189 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58eeeb6f-6db0-4bfe-a7c1-feb94f2d43e4 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:29.678368 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1449/2893] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 20:09:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 20:09:29.682300 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b39ecb-a35b-4105-a523-8ab7de9b53b9 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:29.692116 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b39ecb-a35b-4105-a523-8ab7de9b53b9 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:29.696915 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f273827-b809-40ef-aa1f-98fce2695ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:29.697547 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f273827-b809-40ef-aa1f-98fce2695ad5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:29.697874 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f273827-b809-40ef-aa1f-98fce2695ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:29.698096 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f273827-b809-40ef-aa1f-98fce2695ad5 admin admin] PATH_INFO: `/v3/projects/bbeed72942164d838947a5d6a5a5b8e2/users/91a031fe05e8466bab5d9778e4f113d4/roles/163c30c3b3c246beb849ce176f02d9da` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:29.714955 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f273827-b809-40ef-aa1f-98fce2695ad5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbeed72942164d838947a5d6a5a5b8e2, user_id=91a031fe05e8466bab5d9778e4f113d4, role_id=163c30c3b3c246beb849ce176f02d9da)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:29.717621 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f273827-b809-40ef-aa1f-98fce2695ad5 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:29.730835 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1445/2894] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 20:09:29 2022] PUT /identity/v3/projects/bbeed72942164d838947a5d6a5a5b8e2/users/91a031fe05e8466bab5d9778e4f113d4/roles/163c30c3b3c246beb849ce176f02d9da => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 20:09:29.734972 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eeeb6f-6db0-4bfe-a7c1-feb94f2d43e4 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:29.745059 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eeeb6f-6db0-4bfe-a7c1-feb94f2d43e4 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:29.801446 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d783f4-65dd-4f3f-8de1-09b23d5f3806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:29.802343 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d783f4-65dd-4f3f-8de1-09b23d5f3806 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:29.802616 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d783f4-65dd-4f3f-8de1-09b23d5f3806 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:29.802883 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d783f4-65dd-4f3f-8de1-09b23d5f3806 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:29.803523 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d783f4-65dd-4f3f-8de1-09b23d5f3806 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:29.805149 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d783f4-65dd-4f3f-8de1-09b23d5f3806 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:29.815604 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1450/2895] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 20:09:29 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 20:09:29.819843 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f273827-b809-40ef-aa1f-98fce2695ad5 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:29.830958 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f273827-b809-40ef-aa1f-98fce2695ad5 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:29.835533 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c87affd-84e2-46e8-aebd-af81530200ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:29.836216 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c87affd-84e2-46e8-aebd-af81530200ca admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:29.836501 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c87affd-84e2-46e8-aebd-af81530200ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:29.836775 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c87affd-84e2-46e8-aebd-af81530200ca admin admin] PATH_INFO: `/v3/projects/bbeed72942164d838947a5d6a5a5b8e2/users/91a031fe05e8466bab5d9778e4f113d4/roles/163c30c3b3c246beb849ce176f02d9da` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:29.838720 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c87affd-84e2-46e8-aebd-af81530200ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbeed72942164d838947a5d6a5a5b8e2, user_id=91a031fe05e8466bab5d9778e4f113d4, role_id=163c30c3b3c246beb849ce176f02d9da)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:29.841407 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c87affd-84e2-46e8-aebd-af81530200ca admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:29.852201 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1446/2896] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 20:09:29 2022] PUT /identity/v3/projects/bbeed72942164d838947a5d6a5a5b8e2/users/91a031fe05e8466bab5d9778e4f113d4/roles/163c30c3b3c246beb849ce176f02d9da => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 20:09:29.862258 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94580968-124c-4fc1-8aee-110988db197d None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:29.862756 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94580968-124c-4fc1-8aee-110988db197d None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:29.862860 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94580968-124c-4fc1-8aee-110988db197d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:29.889315 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-94580968-124c-4fc1-8aee-110988db197d None None] MFA Rules not processed for user `91a031fe05e8466bab5d9778e4f113d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 20:09:29.891057 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-94580968-124c-4fc1-8aee-110988db197d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:09:29.919268 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:09:29.958948 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1451/2897] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 20:09:29 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 20:09:29.970162 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c87affd-84e2-46e8-aebd-af81530200ca admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:29.971985 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-0c87affd-84e2-46e8-aebd-af81530200ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:09:30.028417 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c87affd-84e2-46e8-aebd-af81530200ca admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:30.032740 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee9174e-5f5d-49ba-a59a-a38479f60a60 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e79498873d4ed29a0e19bbea9b9214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:30.033358 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee9174e-5f5d-49ba-a59a-a38479f60a60 service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:30.033631 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee9174e-5f5d-49ba-a59a-a38479f60a60 service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:30.033898 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee9174e-5f5d-49ba-a59a-a38479f60a60 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:30.042621 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee9174e-5f5d-49ba-a59a-a38479f60a60 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:30.044060 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee9174e-5f5d-49ba-a59a-a38479f60a60 service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:30.053832 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1447/2898] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 20:09:29 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 20:09:30.186168 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94580968-124c-4fc1-8aee-110988db197d None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:30.247197 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94580968-124c-4fc1-8aee-110988db197d None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:30.252099 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e17cc6e-6c5a-4ada-9ac3-ccc6962d0c1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:30.252717 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e17cc6e-6c5a-4ada-9ac3-ccc6962d0c1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:30.252989 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e17cc6e-6c5a-4ada-9ac3-ccc6962d0c1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:30.253265 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e17cc6e-6c5a-4ada-9ac3-ccc6962d0c1d admin admin] PATH_INFO: `/v3/users/1d42584c8731425487c20bd4a2cbfc14` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:30.254361 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e17cc6e-6c5a-4ada-9ac3-ccc6962d0c1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d42584c8731425487c20bd4a2cbfc14)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:30.256290 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e17cc6e-6c5a-4ada-9ac3-ccc6962d0c1d admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:30.300723 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-4e17cc6e-6c5a-4ada-9ac3-ccc6962d0c1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d42584c8731425487c20bd4a2cbfc14'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:30.301364 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-4e17cc6e-6c5a-4ada-9ac3-ccc6962d0c1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d42584c8731425487c20bd4a2cbfc14'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:30.308220 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-4e17cc6e-6c5a-4ada-9ac3-ccc6962d0c1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d42584c8731425487c20bd4a2cbfc14'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:30.315934 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-4e17cc6e-6c5a-4ada-9ac3-ccc6962d0c1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d42584c8731425487c20bd4a2cbfc14'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:30.329337 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-4e17cc6e-6c5a-4ada-9ac3-ccc6962d0c1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d42584c8731425487c20bd4a2cbfc14'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:30.335301 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1452/2899] 10.222.0.61 () {58 vars in 1224 bytes} [Mon Aug 22 20:09:30 2022] DELETE /identity/v3/users/1d42584c8731425487c20bd4a2cbfc14 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 20:09:30.341960 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afec6cdf-eabb-4617-9fba-40971cc09e52 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:30.342241 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afec6cdf-eabb-4617-9fba-40971cc09e52 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:30.342519 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afec6cdf-eabb-4617-9fba-40971cc09e52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:30.361355 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-afec6cdf-eabb-4617-9fba-40971cc09e52 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 20:09:30.362551 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-afec6cdf-eabb-4617-9fba-40971cc09e52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:09:30.420352 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1448/2900] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 20:09:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 20:09:30.430608 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e17cc6e-6c5a-4ada-9ac3-ccc6962d0c1d admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:30.432047 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-4e17cc6e-6c5a-4ada-9ac3-ccc6962d0c1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:09:30.478888 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:09:30.491510 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e17cc6e-6c5a-4ada-9ac3-ccc6962d0c1d admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:30.495327 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3107d1f3-a269-420b-85c7-d6fdfa224021 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:30.495962 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3107d1f3-a269-420b-85c7-d6fdfa224021 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:30.496237 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3107d1f3-a269-420b-85c7-d6fdfa224021 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:30.496505 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3107d1f3-a269-420b-85c7-d6fdfa224021 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:30.506020 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3107d1f3-a269-420b-85c7-d6fdfa224021 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:30.507532 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3107d1f3-a269-420b-85c7-d6fdfa224021 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:30.516785 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1453/2901] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 20:09:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 20:09:31.024743 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afec6cdf-eabb-4617-9fba-40971cc09e52 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:31.026235 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-afec6cdf-eabb-4617-9fba-40971cc09e52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:09:31.038987 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afec6cdf-eabb-4617-9fba-40971cc09e52 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:31.043805 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42aa1b8-b012-4e3e-9d73-87fe77413e37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:31.044459 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42aa1b8-b012-4e3e-9d73-87fe77413e37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:31.044717 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42aa1b8-b012-4e3e-9d73-87fe77413e37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:31.044988 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42aa1b8-b012-4e3e-9d73-87fe77413e37 admin admin] PATH_INFO: `/v3/projects/a91f33e61df34a95a72c6e218d3b66f7` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:31.047343 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42aa1b8-b012-4e3e-9d73-87fe77413e37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a91f33e61df34a95a72c6e218d3b66f7)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:31.049308 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42aa1b8-b012-4e3e-9d73-87fe77413e37 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:31.074002 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:09:31.096844 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-b42aa1b8-b012-4e3e-9d73-87fe77413e37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a91f33e61df34a95a72c6e218d3b66f7', 'request_id': 'req-b42aa1b8-b012-4e3e-9d73-87fe77413e37'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:31.103881 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-b42aa1b8-b012-4e3e-9d73-87fe77413e37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a91f33e61df34a95a72c6e218d3b66f7', 'request_id': 'req-b42aa1b8-b012-4e3e-9d73-87fe77413e37'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:31.110598 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-b42aa1b8-b012-4e3e-9d73-87fe77413e37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a91f33e61df34a95a72c6e218d3b66f7', 'request_id': 'req-b42aa1b8-b012-4e3e-9d73-87fe77413e37'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:31.121226 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-b42aa1b8-b012-4e3e-9d73-87fe77413e37 admin admin] The token cache is being invalidate because project a91f33e61df34a95a72c6e218d3b66f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 20:09:31.121531 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-b42aa1b8-b012-4e3e-9d73-87fe77413e37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:31.122209 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-b42aa1b8-b012-4e3e-9d73-87fe77413e37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:31.123723 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1449/2902] 10.222.0.61 () {58 vars in 1233 bytes} [Mon Aug 22 20:09:31 2022] DELETE /identity/v3/projects/a91f33e61df34a95a72c6e218d3b66f7 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 20:09:31.127741 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3107d1f3-a269-420b-85c7-d6fdfa224021 service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:31.137098 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3107d1f3-a269-420b-85c7-d6fdfa224021 service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:31.193994 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a5b6dd-dfd6-4ee4-8234-aef9201d6f3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:31.194622 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a5b6dd-dfd6-4ee4-8234-aef9201d6f3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:31.194912 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a5b6dd-dfd6-4ee4-8234-aef9201d6f3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:31.195178 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a5b6dd-dfd6-4ee4-8234-aef9201d6f3e admin admin] PATH_INFO: `/v3/users/91a031fe05e8466bab5d9778e4f113d4` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:31.196292 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a5b6dd-dfd6-4ee4-8234-aef9201d6f3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=91a031fe05e8466bab5d9778e4f113d4)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:31.198439 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a5b6dd-dfd6-4ee4-8234-aef9201d6f3e admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:31.242792 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-39a5b6dd-dfd6-4ee4-8234-aef9201d6f3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91a031fe05e8466bab5d9778e4f113d4'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:31.243427 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-39a5b6dd-dfd6-4ee4-8234-aef9201d6f3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91a031fe05e8466bab5d9778e4f113d4'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:31.250805 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-39a5b6dd-dfd6-4ee4-8234-aef9201d6f3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91a031fe05e8466bab5d9778e4f113d4'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:31.258023 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-39a5b6dd-dfd6-4ee4-8234-aef9201d6f3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91a031fe05e8466bab5d9778e4f113d4'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:31.269787 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-39a5b6dd-dfd6-4ee4-8234-aef9201d6f3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91a031fe05e8466bab5d9778e4f113d4'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:31.275570 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1454/2903] 10.222.0.61 () {58 vars in 1224 bytes} [Mon Aug 22 20:09:31 2022] DELETE /identity/v3/users/91a031fe05e8466bab5d9778e4f113d4 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 20:09:31.542571 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42aa1b8-b012-4e3e-9d73-87fe77413e37 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:31.544044 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-b42aa1b8-b012-4e3e-9d73-87fe77413e37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:09:31.559175 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42aa1b8-b012-4e3e-9d73-87fe77413e37 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:31.616953 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d93903-4e5e-460c-a660-5cc81f3efc38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:31.617581 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d93903-4e5e-460c-a660-5cc81f3efc38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:31.617859 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d93903-4e5e-460c-a660-5cc81f3efc38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:31.618127 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d93903-4e5e-460c-a660-5cc81f3efc38 admin admin] PATH_INFO: `/v3/projects/bbeed72942164d838947a5d6a5a5b8e2` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:31.619238 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d93903-4e5e-460c-a660-5cc81f3efc38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bbeed72942164d838947a5d6a5a5b8e2)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:31.621155 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d93903-4e5e-460c-a660-5cc81f3efc38 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:31.665762 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-62d93903-4e5e-460c-a660-5cc81f3efc38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbeed72942164d838947a5d6a5a5b8e2', 'request_id': 'req-62d93903-4e5e-460c-a660-5cc81f3efc38'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:31.673062 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-62d93903-4e5e-460c-a660-5cc81f3efc38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbeed72942164d838947a5d6a5a5b8e2', 'request_id': 'req-62d93903-4e5e-460c-a660-5cc81f3efc38'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:31.679247 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-62d93903-4e5e-460c-a660-5cc81f3efc38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbeed72942164d838947a5d6a5a5b8e2', 'request_id': 'req-62d93903-4e5e-460c-a660-5cc81f3efc38'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:31.689454 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-62d93903-4e5e-460c-a660-5cc81f3efc38 admin admin] The token cache is being invalidate because project bbeed72942164d838947a5d6a5a5b8e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 20:09:31.689740 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-62d93903-4e5e-460c-a660-5cc81f3efc38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:31.690367 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-62d93903-4e5e-460c-a660-5cc81f3efc38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:31.691847 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1450/2904] 10.222.0.61 () {58 vars in 1233 bytes} [Mon Aug 22 20:09:31 2022] DELETE /identity/v3/projects/bbeed72942164d838947a5d6a5a5b8e2 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 20:09:31.696991 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d53664-db53-487c-abb6-3699d549ee90 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:31.697279 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d53664-db53-487c-abb6-3699d549ee90 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:31.697586 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d53664-db53-487c-abb6-3699d549ee90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:31.716635 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-09d53664-db53-487c-abb6-3699d549ee90 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 20:09:31.718402 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-09d53664-db53-487c-abb6-3699d549ee90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:09:31.776320 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1455/2905] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 20:09:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 20:09:31.785587 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03de9d65-caec-4a4a-97fa-9865a08ee91e None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:31.785900 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03de9d65-caec-4a4a-97fa-9865a08ee91e None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:31.786171 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03de9d65-caec-4a4a-97fa-9865a08ee91e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:31.806837 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-03de9d65-caec-4a4a-97fa-9865a08ee91e None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 20:09:31.808087 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-03de9d65-caec-4a4a-97fa-9865a08ee91e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:09:31.816294 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1451/2906] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 20:09:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 20:09:31.820782 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d53664-db53-487c-abb6-3699d549ee90 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:31.830993 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d53664-db53-487c-abb6-3699d549ee90 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:31.835524 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b191aeb-b31d-4058-8add-e3ebe93a5081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:31.836153 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b191aeb-b31d-4058-8add-e3ebe93a5081 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:31.836435 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b191aeb-b31d-4058-8add-e3ebe93a5081 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:31.836708 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b191aeb-b31d-4058-8add-e3ebe93a5081 admin admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:31.837352 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b191aeb-b31d-4058-8add-e3ebe93a5081 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 20:09:31.837652 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b191aeb-b31d-4058-8add-e3ebe93a5081 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:31.839241 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b191aeb-b31d-4058-8add-e3ebe93a5081 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:31.840712 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:09:31.853792 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1456/2907] 10.222.0.61 () {58 vars in 1153 bytes} [Mon Aug 22 20:09:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 20:09:31.858257 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03de9d65-caec-4a4a-97fa-9865a08ee91e None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:31.868201 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03de9d65-caec-4a4a-97fa-9865a08ee91e None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:31.872836 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37395508-094d-49c3-b9f6-ede447e709e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:31.873452 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37395508-094d-49c3-b9f6-ede447e709e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:31.873729 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37395508-094d-49c3-b9f6-ede447e709e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:31.874001 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37395508-094d-49c3-b9f6-ede447e709e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:31.874633 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37395508-094d-49c3-b9f6-ede447e709e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:31.876606 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37395508-094d-49c3-b9f6-ede447e709e8 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:31.903385 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1452/2908] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 20:09:31 2022] POST /identity/v3/projects => generated 381 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 20:09:31.908189 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b191aeb-b31d-4058-8add-e3ebe93a5081 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:31.918110 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b191aeb-b31d-4058-8add-e3ebe93a5081 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:31.974637 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1531c2d9-e47a-4c8a-8fd4-f557ac08714e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:31.975249 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1531c2d9-e47a-4c8a-8fd4-f557ac08714e admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:31.975522 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1531c2d9-e47a-4c8a-8fd4-f557ac08714e admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:31.975834 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1531c2d9-e47a-4c8a-8fd4-f557ac08714e admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:31.976450 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1531c2d9-e47a-4c8a-8fd4-f557ac08714e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:31.978345 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1531c2d9-e47a-4c8a-8fd4-f557ac08714e admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:31.998997 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1457/2909] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 20:09:31 2022] POST /identity/v3/users => generated 291 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 20:09:32.003706 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37395508-094d-49c3-b9f6-ede447e709e8 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:32.014046 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37395508-094d-49c3-b9f6-ede447e709e8 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:32.018760 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884f3009-a65c-46d7-9465-e68b862557f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:32.019355 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884f3009-a65c-46d7-9465-e68b862557f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:32.019636 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884f3009-a65c-46d7-9465-e68b862557f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:32.019939 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884f3009-a65c-46d7-9465-e68b862557f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:32.020585 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884f3009-a65c-46d7-9465-e68b862557f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:32.022173 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884f3009-a65c-46d7-9465-e68b862557f9 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:32.032552 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1453/2910] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 20:09:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 20:09:32.036227 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1531c2d9-e47a-4c8a-8fd4-f557ac08714e admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:32.046587 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1531c2d9-e47a-4c8a-8fd4-f557ac08714e admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:32.051217 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e954f3-d5ba-45df-a2e7-762903cb39ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:32.051847 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e954f3-d5ba-45df-a2e7-762903cb39ef admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:32.052127 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e954f3-d5ba-45df-a2e7-762903cb39ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:32.052396 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e954f3-d5ba-45df-a2e7-762903cb39ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:32.053034 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e954f3-d5ba-45df-a2e7-762903cb39ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:32.054615 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e954f3-d5ba-45df-a2e7-762903cb39ef admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:32.066487 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1458/2911] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 20:09:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 20:09:32.070463 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884f3009-a65c-46d7-9465-e68b862557f9 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:32.075918 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:09:32.082840 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884f3009-a65c-46d7-9465-e68b862557f9 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:32.088498 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b82f21-78f3-4747-b63c-01e1b11dd684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:32.089174 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b82f21-78f3-4747-b63c-01e1b11dd684 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:32.089480 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b82f21-78f3-4747-b63c-01e1b11dd684 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:32.089755 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b82f21-78f3-4747-b63c-01e1b11dd684 admin admin] PATH_INFO: `/v3/projects/9fadbe1a90534b1884e94807318bc942/users/c41f86515e1146bb9894ce1d717922f0/roles/74a40bc819cb42eea7e966a4b2a016f1` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:32.104059 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b82f21-78f3-4747-b63c-01e1b11dd684 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fadbe1a90534b1884e94807318bc942, user_id=c41f86515e1146bb9894ce1d717922f0, role_id=74a40bc819cb42eea7e966a4b2a016f1)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:32.106687 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b82f21-78f3-4747-b63c-01e1b11dd684 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:32.120288 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1454/2912] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 20:09:32 2022] PUT /identity/v3/projects/9fadbe1a90534b1884e94807318bc942/users/c41f86515e1146bb9894ce1d717922f0/roles/74a40bc819cb42eea7e966a4b2a016f1 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 20:09:32.131247 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9818705e-cb04-4abd-9434-73e975a713c7 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:32.131749 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9818705e-cb04-4abd-9434-73e975a713c7 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:32.131846 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9818705e-cb04-4abd-9434-73e975a713c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:32.160771 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-9818705e-cb04-4abd-9434-73e975a713c7 None None] MFA Rules not processed for user `c41f86515e1146bb9894ce1d717922f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 20:09:32.161992 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-9818705e-cb04-4abd-9434-73e975a713c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:09:32.225375 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1459/2913] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 20:09:32 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 20:09:32.229959 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b82f21-78f3-4747-b63c-01e1b11dd684 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:32.291537 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b82f21-78f3-4747-b63c-01e1b11dd684 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:32.296687 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e9e032-46dc-4a87-9347-e93db1c615e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:32.297274 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e9e032-46dc-4a87-9347-e93db1c615e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:32.297551 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e9e032-46dc-4a87-9347-e93db1c615e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:32.297824 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e9e032-46dc-4a87-9347-e93db1c615e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:32.298438 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e9e032-46dc-4a87-9347-e93db1c615e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:32.300355 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e9e032-46dc-4a87-9347-e93db1c615e8 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:32.326364 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1455/2914] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 20:09:32 2022] POST /identity/v3/projects => generated 379 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 20:09:32.330860 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9818705e-cb04-4abd-9434-73e975a713c7 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:32.342095 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9818705e-cb04-4abd-9434-73e975a713c7 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:32.397445 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c38c672-8e8d-40af-9994-16429580771d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:32.398050 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c38c672-8e8d-40af-9994-16429580771d admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:32.398328 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c38c672-8e8d-40af-9994-16429580771d admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:32.398601 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c38c672-8e8d-40af-9994-16429580771d admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:32.399219 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c38c672-8e8d-40af-9994-16429580771d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:32.401154 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c38c672-8e8d-40af-9994-16429580771d admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:32.410253 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:09:32.423174 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1460/2915] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 20:09:32 2022] POST /identity/v3/users => generated 296 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 20:09:32.427225 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e9e032-46dc-4a87-9347-e93db1c615e8 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:32.437614 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e9e032-46dc-4a87-9347-e93db1c615e8 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:32.442485 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da88137c-c626-4c3c-8681-15993ca77a74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:32.443089 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da88137c-c626-4c3c-8681-15993ca77a74 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:32.443374 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da88137c-c626-4c3c-8681-15993ca77a74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:32.443668 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da88137c-c626-4c3c-8681-15993ca77a74 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:32.444318 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da88137c-c626-4c3c-8681-15993ca77a74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:32.445906 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da88137c-c626-4c3c-8681-15993ca77a74 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:32.456915 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1456/2916] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 20:09:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 20:09:32.460985 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c38c672-8e8d-40af-9994-16429580771d admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:32.471426 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c38c672-8e8d-40af-9994-16429580771d admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:32.476117 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70204b88-4ce5-46f7-8d7a-75748eeb8146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:32.476765 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70204b88-4ce5-46f7-8d7a-75748eeb8146 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:32.477045 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70204b88-4ce5-46f7-8d7a-75748eeb8146 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:32.477316 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70204b88-4ce5-46f7-8d7a-75748eeb8146 admin admin] PATH_INFO: `/v3/projects/a6089d39d53741048d82601d16121589/users/867458c7781543289e3448fd55f8958d/roles/163c30c3b3c246beb849ce176f02d9da` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:32.492401 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70204b88-4ce5-46f7-8d7a-75748eeb8146 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6089d39d53741048d82601d16121589, user_id=867458c7781543289e3448fd55f8958d, role_id=163c30c3b3c246beb849ce176f02d9da)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:32.495019 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70204b88-4ce5-46f7-8d7a-75748eeb8146 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:32.507558 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1461/2917] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 20:09:32 2022] PUT /identity/v3/projects/a6089d39d53741048d82601d16121589/users/867458c7781543289e3448fd55f8958d/roles/163c30c3b3c246beb849ce176f02d9da => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 20:09:32.511798 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da88137c-c626-4c3c-8681-15993ca77a74 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:32.521808 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da88137c-c626-4c3c-8681-15993ca77a74 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:32.577129 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba7d895-4a4b-4c37-b2b1-3dc23ecad787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:32.577751 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba7d895-4a4b-4c37-b2b1-3dc23ecad787 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:32.578026 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba7d895-4a4b-4c37-b2b1-3dc23ecad787 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:32.578294 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba7d895-4a4b-4c37-b2b1-3dc23ecad787 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:32.578921 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba7d895-4a4b-4c37-b2b1-3dc23ecad787 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:32.580526 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba7d895-4a4b-4c37-b2b1-3dc23ecad787 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:32.589907 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1457/2918] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 20:09:32 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 20:09:32.594573 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70204b88-4ce5-46f7-8d7a-75748eeb8146 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:32.604930 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70204b88-4ce5-46f7-8d7a-75748eeb8146 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:32.610161 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315c3552-7e89-4512-98ab-8cb362cde603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:32.610805 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315c3552-7e89-4512-98ab-8cb362cde603 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:32.611084 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315c3552-7e89-4512-98ab-8cb362cde603 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:32.611356 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315c3552-7e89-4512-98ab-8cb362cde603 admin admin] PATH_INFO: `/v3/projects/a6089d39d53741048d82601d16121589/users/867458c7781543289e3448fd55f8958d/roles/163c30c3b3c246beb849ce176f02d9da` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:32.613312 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315c3552-7e89-4512-98ab-8cb362cde603 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6089d39d53741048d82601d16121589, user_id=867458c7781543289e3448fd55f8958d, role_id=163c30c3b3c246beb849ce176f02d9da)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:32.615973 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315c3552-7e89-4512-98ab-8cb362cde603 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:32.625989 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1462/2919] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 20:09:32 2022] PUT /identity/v3/projects/a6089d39d53741048d82601d16121589/users/867458c7781543289e3448fd55f8958d/roles/163c30c3b3c246beb849ce176f02d9da => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 20:09:32.636161 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aef4ce-01c4-43b0-86b3-b79c744c8e31 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:32.636516 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aef4ce-01c4-43b0-86b3-b79c744c8e31 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:32.636796 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aef4ce-01c4-43b0-86b3-b79c744c8e31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:32.664700 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-09aef4ce-01c4-43b0-86b3-b79c744c8e31 None None] MFA Rules not processed for user `867458c7781543289e3448fd55f8958d`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 20:09:32.665903 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-09aef4ce-01c4-43b0-86b3-b79c744c8e31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:09:32.702051 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:09:32.733362 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1458/2920] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 20:09:32 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 20:09:32.745202 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315c3552-7e89-4512-98ab-8cb362cde603 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:32.746932 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-315c3552-7e89-4512-98ab-8cb362cde603 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:09:32.804998 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315c3552-7e89-4512-98ab-8cb362cde603 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:32.808723 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae2f4aa-ca85-43b1-ad3e-bc922c91669b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e79498873d4ed29a0e19bbea9b9214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:32.809265 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae2f4aa-ca85-43b1-ad3e-bc922c91669b service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:32.809541 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae2f4aa-ca85-43b1-ad3e-bc922c91669b service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:32.809812 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae2f4aa-ca85-43b1-ad3e-bc922c91669b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:32.819048 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae2f4aa-ca85-43b1-ad3e-bc922c91669b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:32.820501 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae2f4aa-ca85-43b1-ad3e-bc922c91669b service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:32.830685 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1463/2921] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 20:09:32 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 20:09:33.198507 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aef4ce-01c4-43b0-86b3-b79c744c8e31 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:33.259225 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aef4ce-01c4-43b0-86b3-b79c744c8e31 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:33.264584 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984ddc89-cddb-4b5b-a48c-cc56891e5e84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:33.265216 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984ddc89-cddb-4b5b-a48c-cc56891e5e84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:33.265500 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984ddc89-cddb-4b5b-a48c-cc56891e5e84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:33.265771 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984ddc89-cddb-4b5b-a48c-cc56891e5e84 admin admin] PATH_INFO: `/v3/users/c41f86515e1146bb9894ce1d717922f0` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:33.266925 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984ddc89-cddb-4b5b-a48c-cc56891e5e84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c41f86515e1146bb9894ce1d717922f0)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:33.268907 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984ddc89-cddb-4b5b-a48c-cc56891e5e84 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:33.313800 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-984ddc89-cddb-4b5b-a48c-cc56891e5e84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c41f86515e1146bb9894ce1d717922f0'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:33.314371 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-984ddc89-cddb-4b5b-a48c-cc56891e5e84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c41f86515e1146bb9894ce1d717922f0'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:33.321199 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-984ddc89-cddb-4b5b-a48c-cc56891e5e84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c41f86515e1146bb9894ce1d717922f0'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:33.328742 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-984ddc89-cddb-4b5b-a48c-cc56891e5e84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c41f86515e1146bb9894ce1d717922f0'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:33.340918 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-984ddc89-cddb-4b5b-a48c-cc56891e5e84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c41f86515e1146bb9894ce1d717922f0'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:33.347292 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1459/2922] 10.222.0.61 () {58 vars in 1224 bytes} [Mon Aug 22 20:09:33 2022] DELETE /identity/v3/users/c41f86515e1146bb9894ce1d717922f0 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 20:09:33.351827 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25791916-ce80-4dc2-9a48-23384d3f7ff9 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:33.352294 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25791916-ce80-4dc2-9a48-23384d3f7ff9 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:33.352412 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25791916-ce80-4dc2-9a48-23384d3f7ff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:33.371443 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-25791916-ce80-4dc2-9a48-23384d3f7ff9 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 20:09:33.372689 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-25791916-ce80-4dc2-9a48-23384d3f7ff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:09:33.430380 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1464/2923] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 20:09:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 20:09:33.439938 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984ddc89-cddb-4b5b-a48c-cc56891e5e84 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:33.441372 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-984ddc89-cddb-4b5b-a48c-cc56891e5e84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:09:33.498898 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984ddc89-cddb-4b5b-a48c-cc56891e5e84 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:33.502636 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c90084-bccc-4c80-a927-fea62aacc34a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:33.503238 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c90084-bccc-4c80-a927-fea62aacc34a service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:33.503517 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c90084-bccc-4c80-a927-fea62aacc34a service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:33.503820 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c90084-bccc-4c80-a927-fea62aacc34a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:33.510700 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:09:33.513159 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c90084-bccc-4c80-a927-fea62aacc34a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:33.515617 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c90084-bccc-4c80-a927-fea62aacc34a service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:33.526744 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1460/2924] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 20:09:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 20:09:33.798949 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25791916-ce80-4dc2-9a48-23384d3f7ff9 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:33.800486 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-25791916-ce80-4dc2-9a48-23384d3f7ff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:09:33.813581 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25791916-ce80-4dc2-9a48-23384d3f7ff9 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:33.818725 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5059284c-8d99-4f0d-a52f-4fc0fa1e7b14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:33.819441 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5059284c-8d99-4f0d-a52f-4fc0fa1e7b14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:33.819882 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5059284c-8d99-4f0d-a52f-4fc0fa1e7b14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:33.820196 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5059284c-8d99-4f0d-a52f-4fc0fa1e7b14 admin admin] PATH_INFO: `/v3/projects/9fadbe1a90534b1884e94807318bc942` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:33.821414 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5059284c-8d99-4f0d-a52f-4fc0fa1e7b14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9fadbe1a90534b1884e94807318bc942)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:33.823581 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5059284c-8d99-4f0d-a52f-4fc0fa1e7b14 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:33.870034 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-5059284c-8d99-4f0d-a52f-4fc0fa1e7b14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9fadbe1a90534b1884e94807318bc942', 'request_id': 'req-5059284c-8d99-4f0d-a52f-4fc0fa1e7b14'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:33.872622 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:INSERT=5,keystone:SELECT=91,keystone:DELETE=4 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:09:33.879454 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-5059284c-8d99-4f0d-a52f-4fc0fa1e7b14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9fadbe1a90534b1884e94807318bc942', 'request_id': 'req-5059284c-8d99-4f0d-a52f-4fc0fa1e7b14'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:33.887313 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-5059284c-8d99-4f0d-a52f-4fc0fa1e7b14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9fadbe1a90534b1884e94807318bc942', 'request_id': 'req-5059284c-8d99-4f0d-a52f-4fc0fa1e7b14'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:33.898199 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-5059284c-8d99-4f0d-a52f-4fc0fa1e7b14 admin admin] The token cache is being invalidate because project 9fadbe1a90534b1884e94807318bc942 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 20:09:33.898481 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-5059284c-8d99-4f0d-a52f-4fc0fa1e7b14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:33.899110 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-5059284c-8d99-4f0d-a52f-4fc0fa1e7b14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:33.900626 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1465/2925] 10.222.0.61 () {58 vars in 1233 bytes} [Mon Aug 22 20:09:33 2022] DELETE /identity/v3/projects/9fadbe1a90534b1884e94807318bc942 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 20:09:33.904734 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c90084-bccc-4c80-a927-fea62aacc34a service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:33.914399 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c90084-bccc-4c80-a927-fea62aacc34a service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:33.970914 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6f60d5-4e99-452c-95fd-db0bd1241a07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:33.971541 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6f60d5-4e99-452c-95fd-db0bd1241a07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:33.971849 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6f60d5-4e99-452c-95fd-db0bd1241a07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:33.972124 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6f60d5-4e99-452c-95fd-db0bd1241a07 admin admin] PATH_INFO: `/v3/users/867458c7781543289e3448fd55f8958d` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:33.973247 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6f60d5-4e99-452c-95fd-db0bd1241a07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=867458c7781543289e3448fd55f8958d)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:33.975141 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6f60d5-4e99-452c-95fd-db0bd1241a07 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:34.025424 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-7e6f60d5-4e99-452c-95fd-db0bd1241a07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '867458c7781543289e3448fd55f8958d'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:34.026058 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-7e6f60d5-4e99-452c-95fd-db0bd1241a07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '867458c7781543289e3448fd55f8958d'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:34.032924 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-7e6f60d5-4e99-452c-95fd-db0bd1241a07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '867458c7781543289e3448fd55f8958d'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:34.040080 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-7e6f60d5-4e99-452c-95fd-db0bd1241a07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '867458c7781543289e3448fd55f8958d'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:34.051796 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-7e6f60d5-4e99-452c-95fd-db0bd1241a07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '867458c7781543289e3448fd55f8958d'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:34.057520 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1461/2926] 10.222.0.61 () {58 vars in 1224 bytes} [Mon Aug 22 20:09:33 2022] DELETE /identity/v3/users/867458c7781543289e3448fd55f8958d => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 20:09:34.321276 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5059284c-8d99-4f0d-a52f-4fc0fa1e7b14 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:34.323101 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-5059284c-8d99-4f0d-a52f-4fc0fa1e7b14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:09:34.337255 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5059284c-8d99-4f0d-a52f-4fc0fa1e7b14 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:34.394429 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eaedf6-2bd0-4ae9-987a-0323fc89ebb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:34.395096 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eaedf6-2bd0-4ae9-987a-0323fc89ebb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:34.395304 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eaedf6-2bd0-4ae9-987a-0323fc89ebb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:34.395570 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eaedf6-2bd0-4ae9-987a-0323fc89ebb1 admin admin] PATH_INFO: `/v3/projects/a6089d39d53741048d82601d16121589` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:34.396768 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38eaedf6-2bd0-4ae9-987a-0323fc89ebb1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6089d39d53741048d82601d16121589)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:34.398588 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38eaedf6-2bd0-4ae9-987a-0323fc89ebb1 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:34.445334 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-38eaedf6-2bd0-4ae9-987a-0323fc89ebb1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6089d39d53741048d82601d16121589', 'request_id': 'req-38eaedf6-2bd0-4ae9-987a-0323fc89ebb1'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:34.452804 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-38eaedf6-2bd0-4ae9-987a-0323fc89ebb1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6089d39d53741048d82601d16121589', 'request_id': 'req-38eaedf6-2bd0-4ae9-987a-0323fc89ebb1'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:34.458892 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-38eaedf6-2bd0-4ae9-987a-0323fc89ebb1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6089d39d53741048d82601d16121589', 'request_id': 'req-38eaedf6-2bd0-4ae9-987a-0323fc89ebb1'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:34.468930 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-38eaedf6-2bd0-4ae9-987a-0323fc89ebb1 admin admin] The token cache is being invalidate because project a6089d39d53741048d82601d16121589 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 20:09:34.469216 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-38eaedf6-2bd0-4ae9-987a-0323fc89ebb1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:34.469834 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-38eaedf6-2bd0-4ae9-987a-0323fc89ebb1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:34.471295 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1466/2927] 10.222.0.61 () {58 vars in 1233 bytes} [Mon Aug 22 20:09:34 2022] DELETE /identity/v3/projects/a6089d39d53741048d82601d16121589 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 20:09:34.476172 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0062e7b3-21ab-468e-8be7-f165798186b6 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:34.476653 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0062e7b3-21ab-468e-8be7-f165798186b6 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:34.476750 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0062e7b3-21ab-468e-8be7-f165798186b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:34.496250 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-0062e7b3-21ab-468e-8be7-f165798186b6 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 20:09:34.497454 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-0062e7b3-21ab-468e-8be7-f165798186b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:09:34.556117 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1462/2928] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 20:09:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 20:09:34.564210 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547a0762-9cc7-4128-946b-93bf2c8642f4 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:34.564496 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547a0762-9cc7-4128-946b-93bf2c8642f4 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:34.564770 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547a0762-9cc7-4128-946b-93bf2c8642f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:34.585221 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-547a0762-9cc7-4128-946b-93bf2c8642f4 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 20:09:34.586580 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-547a0762-9cc7-4128-946b-93bf2c8642f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:09:34.595160 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1467/2929] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 20:09:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 20:09:34.599325 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0062e7b3-21ab-468e-8be7-f165798186b6 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:34.609858 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0062e7b3-21ab-468e-8be7-f165798186b6 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:34.614378 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54639596-4367-4263-9138-6d0dfd889ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:34.614967 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54639596-4367-4263-9138-6d0dfd889ed2 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:34.615245 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54639596-4367-4263-9138-6d0dfd889ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:34.615511 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54639596-4367-4263-9138-6d0dfd889ed2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:34.616186 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54639596-4367-4263-9138-6d0dfd889ed2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 20:09:34.616482 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54639596-4367-4263-9138-6d0dfd889ed2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:34.618051 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54639596-4367-4263-9138-6d0dfd889ed2 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:34.629423 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1463/2930] 10.222.0.61 () {58 vars in 1153 bytes} [Mon Aug 22 20:09:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 20:09:34.633602 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547a0762-9cc7-4128-946b-93bf2c8642f4 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:34.642674 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547a0762-9cc7-4128-946b-93bf2c8642f4 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:34.647380 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bd5100-d6f6-4a18-8f0b-dec60e3b734c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:34.647932 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bd5100-d6f6-4a18-8f0b-dec60e3b734c admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:34.648203 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bd5100-d6f6-4a18-8f0b-dec60e3b734c admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:34.648466 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bd5100-d6f6-4a18-8f0b-dec60e3b734c admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:34.649078 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bd5100-d6f6-4a18-8f0b-dec60e3b734c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:34.650941 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bd5100-d6f6-4a18-8f0b-dec60e3b734c admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:34.677579 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1468/2931] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 20:09:34 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 20:09:34.681623 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54639596-4367-4263-9138-6d0dfd889ed2 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:34.686834 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:09:34.692960 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54639596-4367-4263-9138-6d0dfd889ed2 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:34.748719 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3a76ba-fbd6-4b0a-992e-4250ebedd08a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:34.749352 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3a76ba-fbd6-4b0a-992e-4250ebedd08a admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:34.749625 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3a76ba-fbd6-4b0a-992e-4250ebedd08a admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:34.749892 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3a76ba-fbd6-4b0a-992e-4250ebedd08a admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:34.750513 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3a76ba-fbd6-4b0a-992e-4250ebedd08a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:34.752428 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3a76ba-fbd6-4b0a-992e-4250ebedd08a admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:34.771378 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1464/2932] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 20:09:34 2022] POST /identity/v3/users => generated 293 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 20:09:34.775500 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bd5100-d6f6-4a18-8f0b-dec60e3b734c admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:34.786523 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bd5100-d6f6-4a18-8f0b-dec60e3b734c admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:34.791189 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7498161d-d76c-4025-990f-98afa1762037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:34.791827 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7498161d-d76c-4025-990f-98afa1762037 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:34.792106 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7498161d-d76c-4025-990f-98afa1762037 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:34.792377 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7498161d-d76c-4025-990f-98afa1762037 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:34.793018 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7498161d-d76c-4025-990f-98afa1762037 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:34.794622 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7498161d-d76c-4025-990f-98afa1762037 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:34.806124 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1469/2933] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 20:09:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 20:09:34.810112 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3a76ba-fbd6-4b0a-992e-4250ebedd08a admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:34.820231 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3a76ba-fbd6-4b0a-992e-4250ebedd08a admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:34.825025 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd449139-d081-407f-809c-80f30531dd21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:34.825608 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd449139-d081-407f-809c-80f30531dd21 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:34.825884 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd449139-d081-407f-809c-80f30531dd21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:34.826150 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd449139-d081-407f-809c-80f30531dd21 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:34.826782 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd449139-d081-407f-809c-80f30531dd21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:34.828439 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd449139-d081-407f-809c-80f30531dd21 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:34.838625 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1465/2934] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 20:09:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 20:09:34.842657 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7498161d-d76c-4025-990f-98afa1762037 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:34.853228 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7498161d-d76c-4025-990f-98afa1762037 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:34.857803 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d8d2da-3653-4fcd-a2a5-946907634c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:34.858487 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d8d2da-3653-4fcd-a2a5-946907634c74 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:34.859259 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d8d2da-3653-4fcd-a2a5-946907634c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:34.859557 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d8d2da-3653-4fcd-a2a5-946907634c74 admin admin] PATH_INFO: `/v3/projects/bd188f8c34af457485f58e03ba39bdf9/users/ea82af24a7f84fad910d991978270230/roles/74a40bc819cb42eea7e966a4b2a016f1` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:34.874293 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d8d2da-3653-4fcd-a2a5-946907634c74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd188f8c34af457485f58e03ba39bdf9, user_id=ea82af24a7f84fad910d991978270230, role_id=74a40bc819cb42eea7e966a4b2a016f1)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:34.876969 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d8d2da-3653-4fcd-a2a5-946907634c74 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:34.889527 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1470/2935] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 20:09:34 2022] PUT /identity/v3/projects/bd188f8c34af457485f58e03ba39bdf9/users/ea82af24a7f84fad910d991978270230/roles/74a40bc819cb42eea7e966a4b2a016f1 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 20:09:34.900310 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e47957-bbae-4ecb-992e-38ebb50c8781 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:34.900546 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e47957-bbae-4ecb-992e-38ebb50c8781 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:34.900825 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e47957-bbae-4ecb-992e-38ebb50c8781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:34.927939 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-94e47957-bbae-4ecb-992e-38ebb50c8781 None None] MFA Rules not processed for user `ea82af24a7f84fad910d991978270230`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 20:09:34.929154 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-94e47957-bbae-4ecb-992e-38ebb50c8781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:09:34.992282 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1466/2936] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 20:09:34 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 20:09:34.996662 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d8d2da-3653-4fcd-a2a5-946907634c74 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:35.006284 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:DELETE=10,keystone:SELECT=87,keystone:INSERT=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:09:35.060031 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d8d2da-3653-4fcd-a2a5-946907634c74 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:35.064741 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0a5556-3208-47c6-8c95-3e07afd74dc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:35.065340 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0a5556-3208-47c6-8c95-3e07afd74dc2 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:35.065618 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0a5556-3208-47c6-8c95-3e07afd74dc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:35.065892 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0a5556-3208-47c6-8c95-3e07afd74dc2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:35.066519 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0a5556-3208-47c6-8c95-3e07afd74dc2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:35.068434 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0a5556-3208-47c6-8c95-3e07afd74dc2 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:35.094871 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1471/2937] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 20:09:34 2022] POST /identity/v3/projects => generated 383 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 20:09:35.099527 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e47957-bbae-4ecb-992e-38ebb50c8781 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:35.111043 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e47957-bbae-4ecb-992e-38ebb50c8781 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:35.167120 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08337366-ec4f-4d55-911e-e6c76fc3e65a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:35.167667 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08337366-ec4f-4d55-911e-e6c76fc3e65a admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:35.167958 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08337366-ec4f-4d55-911e-e6c76fc3e65a admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:35.168233 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08337366-ec4f-4d55-911e-e6c76fc3e65a admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:35.168968 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08337366-ec4f-4d55-911e-e6c76fc3e65a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:35.170751 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08337366-ec4f-4d55-911e-e6c76fc3e65a admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:35.188471 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:09:35.194455 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1467/2938] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 20:09:35 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 20:09:35.198601 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0a5556-3208-47c6-8c95-3e07afd74dc2 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:35.209620 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0a5556-3208-47c6-8c95-3e07afd74dc2 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:35.214324 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf6e24d-419a-4a5e-b24a-77f028c4e93e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:35.214918 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf6e24d-419a-4a5e-b24a-77f028c4e93e admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:35.215200 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf6e24d-419a-4a5e-b24a-77f028c4e93e admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:35.215564 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf6e24d-419a-4a5e-b24a-77f028c4e93e admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:35.216255 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf6e24d-419a-4a5e-b24a-77f028c4e93e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:35.218462 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf6e24d-419a-4a5e-b24a-77f028c4e93e admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:35.229032 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1472/2939] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 20:09:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 20:09:35.232917 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08337366-ec4f-4d55-911e-e6c76fc3e65a admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:35.243154 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08337366-ec4f-4d55-911e-e6c76fc3e65a admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:35.247955 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d357ebf9-05fe-4fd6-9e0b-c7de3f2d6ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:35.248549 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d357ebf9-05fe-4fd6-9e0b-c7de3f2d6ef7 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:35.248824 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d357ebf9-05fe-4fd6-9e0b-c7de3f2d6ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:35.249598 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d357ebf9-05fe-4fd6-9e0b-c7de3f2d6ef7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:35.250234 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d357ebf9-05fe-4fd6-9e0b-c7de3f2d6ef7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:35.251838 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d357ebf9-05fe-4fd6-9e0b-c7de3f2d6ef7 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:35.262176 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1468/2940] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 20:09:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 20:09:35.267019 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf6e24d-419a-4a5e-b24a-77f028c4e93e admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:35.276720 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf6e24d-419a-4a5e-b24a-77f028c4e93e admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:35.281231 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b631fe5-e77c-4c93-b962-921a291c9842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:35.281864 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b631fe5-e77c-4c93-b962-921a291c9842 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:35.282135 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b631fe5-e77c-4c93-b962-921a291c9842 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:35.282401 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b631fe5-e77c-4c93-b962-921a291c9842 admin admin] PATH_INFO: `/v3/projects/fdec933e50f245a5887bc57545ec14e7/users/c10de21288ef412288a72b3dc09087f7/roles/74a40bc819cb42eea7e966a4b2a016f1` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:35.297125 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b631fe5-e77c-4c93-b962-921a291c9842 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdec933e50f245a5887bc57545ec14e7, user_id=c10de21288ef412288a72b3dc09087f7, role_id=74a40bc819cb42eea7e966a4b2a016f1)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:35.299794 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b631fe5-e77c-4c93-b962-921a291c9842 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:35.313126 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1473/2941] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 20:09:35 2022] PUT /identity/v3/projects/fdec933e50f245a5887bc57545ec14e7/users/c10de21288ef412288a72b3dc09087f7/roles/74a40bc819cb42eea7e966a4b2a016f1 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 20:09:35.384376 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ed11af-7c20-4dc0-af69-f6baba43026f None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:35.384660 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ed11af-7c20-4dc0-af69-f6baba43026f None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:35.384941 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ed11af-7c20-4dc0-af69-f6baba43026f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:35.412640 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-84ed11af-7c20-4dc0-af69-f6baba43026f None None] MFA Rules not processed for user `c10de21288ef412288a72b3dc09087f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 20:09:35.413844 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-84ed11af-7c20-4dc0-af69-f6baba43026f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:09:35.476594 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1469/2942] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 20:09:35 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 20:09:35.481046 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b631fe5-e77c-4c93-b962-921a291c9842 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:35.541551 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b631fe5-e77c-4c93-b962-921a291c9842 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:35.546718 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4955b18a-e886-46b6-a7f1-0d649d0baaa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:35.547314 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4955b18a-e886-46b6-a7f1-0d649d0baaa2 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:35.547589 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4955b18a-e886-46b6-a7f1-0d649d0baaa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:35.547884 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4955b18a-e886-46b6-a7f1-0d649d0baaa2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:35.548493 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4955b18a-e886-46b6-a7f1-0d649d0baaa2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:35.550400 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4955b18a-e886-46b6-a7f1-0d649d0baaa2 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:35.575634 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1474/2943] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 20:09:35 2022] POST /identity/v3/projects => generated 385 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 20:09:35.579579 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ed11af-7c20-4dc0-af69-f6baba43026f None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:35.590098 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ed11af-7c20-4dc0-af69-f6baba43026f None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:35.646113 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157dc5d5-fe8d-4394-b972-839a8b444ad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:35.646710 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157dc5d5-fe8d-4394-b972-839a8b444ad9 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:35.646983 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157dc5d5-fe8d-4394-b972-839a8b444ad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:35.647249 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157dc5d5-fe8d-4394-b972-839a8b444ad9 admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:35.647895 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157dc5d5-fe8d-4394-b972-839a8b444ad9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:35.649784 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157dc5d5-fe8d-4394-b972-839a8b444ad9 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:35.670573 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1470/2944] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 20:09:35 2022] POST /identity/v3/users => generated 299 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 20:09:35.674560 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4955b18a-e886-46b6-a7f1-0d649d0baaa2 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:35.685181 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4955b18a-e886-46b6-a7f1-0d649d0baaa2 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:35.689888 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a3bdb4-fbe8-48bf-8796-d364eeb9c2ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:35.690486 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a3bdb4-fbe8-48bf-8796-d364eeb9c2ec admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:35.690766 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a3bdb4-fbe8-48bf-8796-d364eeb9c2ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:35.691039 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a3bdb4-fbe8-48bf-8796-d364eeb9c2ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:35.691753 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a3bdb4-fbe8-48bf-8796-d364eeb9c2ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:35.693359 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a3bdb4-fbe8-48bf-8796-d364eeb9c2ec admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:35.704664 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1475/2945] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 20:09:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 20:09:35.708699 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157dc5d5-fe8d-4394-b972-839a8b444ad9 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:35.719308 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157dc5d5-fe8d-4394-b972-839a8b444ad9 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:35.723929 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ce143f-f15a-48f8-a523-678fc31ac3ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:35.724569 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ce143f-f15a-48f8-a523-678fc31ac3ac admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:35.724853 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ce143f-f15a-48f8-a523-678fc31ac3ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:35.725126 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ce143f-f15a-48f8-a523-678fc31ac3ac admin admin] PATH_INFO: `/v3/projects/c8a0b99a52d649959bbde0d378cb645a/users/4556cd81adc14c59b37c6981738d010f/roles/163c30c3b3c246beb849ce176f02d9da` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:35.739441 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ce143f-f15a-48f8-a523-678fc31ac3ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8a0b99a52d649959bbde0d378cb645a, user_id=4556cd81adc14c59b37c6981738d010f, role_id=163c30c3b3c246beb849ce176f02d9da)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:35.742078 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ce143f-f15a-48f8-a523-678fc31ac3ac admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:35.754528 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1471/2946] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 20:09:35 2022] PUT /identity/v3/projects/c8a0b99a52d649959bbde0d378cb645a/users/4556cd81adc14c59b37c6981738d010f/roles/163c30c3b3c246beb849ce176f02d9da => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 20:09:35.758348 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a3bdb4-fbe8-48bf-8796-d364eeb9c2ec admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:35.768824 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a3bdb4-fbe8-48bf-8796-d364eeb9c2ec admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:35.797538 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:09:35.827765 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f64c05a-e9e8-4b10-b0e0-3c8a94ca8da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:35.828375 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f64c05a-e9e8-4b10-b0e0-3c8a94ca8da2 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:35.828655 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f64c05a-e9e8-4b10-b0e0-3c8a94ca8da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:35.828927 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f64c05a-e9e8-4b10-b0e0-3c8a94ca8da2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:35.829566 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f64c05a-e9e8-4b10-b0e0-3c8a94ca8da2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:35.831160 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f64c05a-e9e8-4b10-b0e0-3c8a94ca8da2 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:35.840913 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1476/2947] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 20:09:35 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 20:09:35.845713 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ce143f-f15a-48f8-a523-678fc31ac3ac admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:35.855988 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ce143f-f15a-48f8-a523-678fc31ac3ac admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:35.860696 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da70c756-3580-4183-aecb-375715e2f99e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:35.861343 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da70c756-3580-4183-aecb-375715e2f99e admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:35.861616 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da70c756-3580-4183-aecb-375715e2f99e admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:35.861887 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da70c756-3580-4183-aecb-375715e2f99e admin admin] PATH_INFO: `/v3/projects/c8a0b99a52d649959bbde0d378cb645a/users/4556cd81adc14c59b37c6981738d010f/roles/163c30c3b3c246beb849ce176f02d9da` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:35.863860 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da70c756-3580-4183-aecb-375715e2f99e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8a0b99a52d649959bbde0d378cb645a, user_id=4556cd81adc14c59b37c6981738d010f, role_id=163c30c3b3c246beb849ce176f02d9da)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:35.866466 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da70c756-3580-4183-aecb-375715e2f99e admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:35.876625 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1472/2948] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 20:09:35 2022] PUT /identity/v3/projects/c8a0b99a52d649959bbde0d378cb645a/users/4556cd81adc14c59b37c6981738d010f/roles/163c30c3b3c246beb849ce176f02d9da => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 20:09:35.886334 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d54ed86-c2b2-4afd-b4f5-f0a8f36dc538 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:35.886620 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d54ed86-c2b2-4afd-b4f5-f0a8f36dc538 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:35.886894 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d54ed86-c2b2-4afd-b4f5-f0a8f36dc538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:35.914941 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-2d54ed86-c2b2-4afd-b4f5-f0a8f36dc538 None None] MFA Rules not processed for user `4556cd81adc14c59b37c6981738d010f`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 20:09:35.916180 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-2d54ed86-c2b2-4afd-b4f5-f0a8f36dc538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:09:35.982489 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1477/2949] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 20:09:35 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 20:09:35.993057 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b53443-6f6d-44bd-9157-d4fb21ee3f8c None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:35.993286 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b53443-6f6d-44bd-9157-d4fb21ee3f8c None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:35.993558 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b53443-6f6d-44bd-9157-d4fb21ee3f8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:36.032737 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-08b53443-6f6d-44bd-9157-d4fb21ee3f8c None None] MFA Rules not processed for user `56e79498873d4ed29a0e19bbea9b9214`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 20:09:36.033941 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-08b53443-6f6d-44bd-9157-d4fb21ee3f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:09:36.046464 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:09:36.085031 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1473/2950] 10.222.0.61 () {58 vars in 1061 bytes} [Mon Aug 22 20:09:35 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 20:09:36.090950 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d54ed86-c2b2-4afd-b4f5-f0a8f36dc538 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:36.100637 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d54ed86-c2b2-4afd-b4f5-f0a8f36dc538 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:36.104189 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1d87b4-e714-41bc-804a-76ce4cde6d1d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e79498873d4ed29a0e19bbea9b9214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:36.104779 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1d87b4-e714-41bc-804a-76ce4cde6d1d service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:36.105054 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1d87b4-e714-41bc-804a-76ce4cde6d1d service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:36.105323 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1d87b4-e714-41bc-804a-76ce4cde6d1d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:36.161476 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1d87b4-e714-41bc-804a-76ce4cde6d1d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:36.162932 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1d87b4-e714-41bc-804a-76ce4cde6d1d service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:36.172593 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1478/2951] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 20:09:36 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 20:09:37.660740 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b53443-6f6d-44bd-9157-d4fb21ee3f8c None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:37.662489 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-08b53443-6f6d-44bd-9157-d4fb21ee3f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:09:37.673653 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b53443-6f6d-44bd-9157-d4fb21ee3f8c None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:37.677311 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f71708e-ec2f-48b0-b52c-0ee0affd3ce5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e79498873d4ed29a0e19bbea9b9214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:37.677909 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f71708e-ec2f-48b0-b52c-0ee0affd3ce5 service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:37.678187 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f71708e-ec2f-48b0-b52c-0ee0affd3ce5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:37.678460 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f71708e-ec2f-48b0-b52c-0ee0affd3ce5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:37.735389 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f71708e-ec2f-48b0-b52c-0ee0affd3ce5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:37.736768 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f71708e-ec2f-48b0-b52c-0ee0affd3ce5 service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:37.745686 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1474/2952] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 20:09:37 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 20:09:38.266762 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1d87b4-e714-41bc-804a-76ce4cde6d1d service cinder] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:38.275297 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1d87b4-e714-41bc-804a-76ce4cde6d1d service cinder] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:38.278942 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b112f7-1f8b-40d5-ac49-bcf8bd7f1e07 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e79498873d4ed29a0e19bbea9b9214', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:38.279537 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b112f7-1f8b-40d5-ac49-bcf8bd7f1e07 service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:38.279860 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b112f7-1f8b-40d5-ac49-bcf8bd7f1e07 service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:38.280136 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b112f7-1f8b-40d5-ac49-bcf8bd7f1e07 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:38.288950 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b112f7-1f8b-40d5-ac49-bcf8bd7f1e07 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:38.290376 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b112f7-1f8b-40d5-ac49-bcf8bd7f1e07 service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:38.299809 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1479/2953] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 20:09:38 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 20:09:42.140319 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f71708e-ec2f-48b0-b52c-0ee0affd3ce5 service cinder] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:42.204480 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f71708e-ec2f-48b0-b52c-0ee0affd3ce5 service cinder] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:42.208983 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f0b0d3-fd93-46ca-963a-aa9fdb9f6173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:42.209606 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f0b0d3-fd93-46ca-963a-aa9fdb9f6173 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:42.209884 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f0b0d3-fd93-46ca-963a-aa9fdb9f6173 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:42.210154 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f0b0d3-fd93-46ca-963a-aa9fdb9f6173 admin admin] PATH_INFO: `/v3/users/ea82af24a7f84fad910d991978270230` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:42.211264 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f0b0d3-fd93-46ca-963a-aa9fdb9f6173 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea82af24a7f84fad910d991978270230)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:42.213202 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f0b0d3-fd93-46ca-963a-aa9fdb9f6173 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:42.258918 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-47f0b0d3-fd93-46ca-963a-aa9fdb9f6173 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea82af24a7f84fad910d991978270230'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:42.259559 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-47f0b0d3-fd93-46ca-963a-aa9fdb9f6173 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea82af24a7f84fad910d991978270230'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:42.267223 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-47f0b0d3-fd93-46ca-963a-aa9fdb9f6173 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea82af24a7f84fad910d991978270230'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:42.275089 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-47f0b0d3-fd93-46ca-963a-aa9fdb9f6173 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea82af24a7f84fad910d991978270230'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:42.279771 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:09:42.289758 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-47f0b0d3-fd93-46ca-963a-aa9fdb9f6173 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea82af24a7f84fad910d991978270230'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:42.295480 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1475/2954] 10.222.0.61 () {58 vars in 1224 bytes} [Mon Aug 22 20:09:42 2022] DELETE /identity/v3/users/ea82af24a7f84fad910d991978270230 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 20:09:42.300287 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2fa5f1-6ea5-47bb-9110-214f96dc0027 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:42.300780 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2fa5f1-6ea5-47bb-9110-214f96dc0027 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:42.300901 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2fa5f1-6ea5-47bb-9110-214f96dc0027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:42.320507 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-dd2fa5f1-6ea5-47bb-9110-214f96dc0027 None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 20:09:42.321953 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-dd2fa5f1-6ea5-47bb-9110-214f96dc0027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:09:42.353114 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:09:42.380008 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1480/2955] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 20:09:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 20:09:42.390323 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f0b0d3-fd93-46ca-963a-aa9fdb9f6173 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:42.391847 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-47f0b0d3-fd93-46ca-963a-aa9fdb9f6173 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:09:42.448066 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f0b0d3-fd93-46ca-963a-aa9fdb9f6173 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:42.451690 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568c5b77-013c-4ea3-88cc-a267b1abe0f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:42.452286 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568c5b77-013c-4ea3-88cc-a267b1abe0f1 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:42.452563 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568c5b77-013c-4ea3-88cc-a267b1abe0f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:42.452841 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568c5b77-013c-4ea3-88cc-a267b1abe0f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:42.461683 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568c5b77-013c-4ea3-88cc-a267b1abe0f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:42.463133 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568c5b77-013c-4ea3-88cc-a267b1abe0f1 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:42.472589 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1476/2956] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 20:09:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 20:09:42.735316 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2fa5f1-6ea5-47bb-9110-214f96dc0027 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:42.736832 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-dd2fa5f1-6ea5-47bb-9110-214f96dc0027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:09:42.750212 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2fa5f1-6ea5-47bb-9110-214f96dc0027 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:42.755076 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40576f2c-efac-4ba5-a57f-4ff610746d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:42.755824 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40576f2c-efac-4ba5-a57f-4ff610746d03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:42.756397 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40576f2c-efac-4ba5-a57f-4ff610746d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:42.756792 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40576f2c-efac-4ba5-a57f-4ff610746d03 admin admin] PATH_INFO: `/v3/projects/bd188f8c34af457485f58e03ba39bdf9` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:42.758056 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40576f2c-efac-4ba5-a57f-4ff610746d03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd188f8c34af457485f58e03ba39bdf9)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:42.760341 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40576f2c-efac-4ba5-a57f-4ff610746d03 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:42.806011 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-40576f2c-efac-4ba5-a57f-4ff610746d03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd188f8c34af457485f58e03ba39bdf9', 'request_id': 'req-40576f2c-efac-4ba5-a57f-4ff610746d03'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:42.812590 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-40576f2c-efac-4ba5-a57f-4ff610746d03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd188f8c34af457485f58e03ba39bdf9', 'request_id': 'req-40576f2c-efac-4ba5-a57f-4ff610746d03'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:42.818866 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-40576f2c-efac-4ba5-a57f-4ff610746d03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd188f8c34af457485f58e03ba39bdf9', 'request_id': 'req-40576f2c-efac-4ba5-a57f-4ff610746d03'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:42.829247 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-40576f2c-efac-4ba5-a57f-4ff610746d03 admin admin] The token cache is being invalidate because project bd188f8c34af457485f58e03ba39bdf9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 20:09:42.829528 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-40576f2c-efac-4ba5-a57f-4ff610746d03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:42.830147 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-40576f2c-efac-4ba5-a57f-4ff610746d03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:42.831599 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1481/2957] 10.222.0.61 () {58 vars in 1233 bytes} [Mon Aug 22 20:09:42 2022] DELETE /identity/v3/projects/bd188f8c34af457485f58e03ba39bdf9 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 20:09:42.835503 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568c5b77-013c-4ea3-88cc-a267b1abe0f1 service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:42.846132 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568c5b77-013c-4ea3-88cc-a267b1abe0f1 service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:42.900792 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3d8302-fa95-46e6-8a32-c4a910d8129c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:42.901373 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3d8302-fa95-46e6-8a32-c4a910d8129c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:42.901673 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3d8302-fa95-46e6-8a32-c4a910d8129c admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:42.901942 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3d8302-fa95-46e6-8a32-c4a910d8129c admin admin] PATH_INFO: `/v3/users/c10de21288ef412288a72b3dc09087f7` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:42.903104 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3d8302-fa95-46e6-8a32-c4a910d8129c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c10de21288ef412288a72b3dc09087f7)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:42.907171 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3d8302-fa95-46e6-8a32-c4a910d8129c admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:42.952830 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-ba3d8302-fa95-46e6-8a32-c4a910d8129c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c10de21288ef412288a72b3dc09087f7'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:42.953465 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-ba3d8302-fa95-46e6-8a32-c4a910d8129c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c10de21288ef412288a72b3dc09087f7'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:42.960178 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-ba3d8302-fa95-46e6-8a32-c4a910d8129c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c10de21288ef412288a72b3dc09087f7'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:42.966844 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-ba3d8302-fa95-46e6-8a32-c4a910d8129c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c10de21288ef412288a72b3dc09087f7'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:42.978968 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-ba3d8302-fa95-46e6-8a32-c4a910d8129c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c10de21288ef412288a72b3dc09087f7'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:42.985228 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1477/2958] 10.222.0.61 () {58 vars in 1224 bytes} [Mon Aug 22 20:09:42 2022] DELETE /identity/v3/users/c10de21288ef412288a72b3dc09087f7 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 20:09:43.258911 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40576f2c-efac-4ba5-a57f-4ff610746d03 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:43.260396 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-40576f2c-efac-4ba5-a57f-4ff610746d03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:09:43.274032 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40576f2c-efac-4ba5-a57f-4ff610746d03 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:43.331849 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9d8d36-9dae-4333-b744-55486976b003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:43.332537 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9d8d36-9dae-4333-b744-55486976b003 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:43.332751 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9d8d36-9dae-4333-b744-55486976b003 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:43.333024 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9d8d36-9dae-4333-b744-55486976b003 admin admin] PATH_INFO: `/v3/projects/fdec933e50f245a5887bc57545ec14e7` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:43.334139 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9d8d36-9dae-4333-b744-55486976b003 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fdec933e50f245a5887bc57545ec14e7)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:43.336070 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9d8d36-9dae-4333-b744-55486976b003 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:43.380988 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-fe9d8d36-9dae-4333-b744-55486976b003 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fdec933e50f245a5887bc57545ec14e7', 'request_id': 'req-fe9d8d36-9dae-4333-b744-55486976b003'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:43.387450 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-fe9d8d36-9dae-4333-b744-55486976b003 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fdec933e50f245a5887bc57545ec14e7', 'request_id': 'req-fe9d8d36-9dae-4333-b744-55486976b003'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:43.394227 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-fe9d8d36-9dae-4333-b744-55486976b003 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fdec933e50f245a5887bc57545ec14e7', 'request_id': 'req-fe9d8d36-9dae-4333-b744-55486976b003'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:43.396187 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:09:43.407105 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-fe9d8d36-9dae-4333-b744-55486976b003 admin admin] The token cache is being invalidate because project fdec933e50f245a5887bc57545ec14e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 20:09:43.407393 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-fe9d8d36-9dae-4333-b744-55486976b003 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:43.408042 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-fe9d8d36-9dae-4333-b744-55486976b003 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:43.409517 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1482/2959] 10.222.0.61 () {58 vars in 1233 bytes} [Mon Aug 22 20:09:43 2022] DELETE /identity/v3/projects/fdec933e50f245a5887bc57545ec14e7 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 20:09:43.413479 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3d8302-fa95-46e6-8a32-c4a910d8129c admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:43.414933 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-ba3d8302-fa95-46e6-8a32-c4a910d8129c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:09:43.428224 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3d8302-fa95-46e6-8a32-c4a910d8129c admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:43.431792 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:09:43.487548 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327b8dab-e2dd-43bb-aa6e-18cc84eb5abd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:43.488233 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327b8dab-e2dd-43bb-aa6e-18cc84eb5abd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:43.488507 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327b8dab-e2dd-43bb-aa6e-18cc84eb5abd admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:43.488774 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327b8dab-e2dd-43bb-aa6e-18cc84eb5abd admin admin] PATH_INFO: `/v3/users/4556cd81adc14c59b37c6981738d010f` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:43.489946 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327b8dab-e2dd-43bb-aa6e-18cc84eb5abd admin admin] RBAC: Authorizing `identity:delete_user(user_id=4556cd81adc14c59b37c6981738d010f)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:43.492080 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327b8dab-e2dd-43bb-aa6e-18cc84eb5abd admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:43.537189 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-327b8dab-e2dd-43bb-aa6e-18cc84eb5abd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4556cd81adc14c59b37c6981738d010f'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:43.537828 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-327b8dab-e2dd-43bb-aa6e-18cc84eb5abd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4556cd81adc14c59b37c6981738d010f'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:43.544803 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-327b8dab-e2dd-43bb-aa6e-18cc84eb5abd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4556cd81adc14c59b37c6981738d010f'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:43.552604 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-327b8dab-e2dd-43bb-aa6e-18cc84eb5abd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4556cd81adc14c59b37c6981738d010f'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:43.564405 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-327b8dab-e2dd-43bb-aa6e-18cc84eb5abd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4556cd81adc14c59b37c6981738d010f'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:43.570361 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1478/2960] 10.222.0.61 () {58 vars in 1224 bytes} [Mon Aug 22 20:09:43 2022] DELETE /identity/v3/users/4556cd81adc14c59b37c6981738d010f => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 20:09:43.837366 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9d8d36-9dae-4333-b744-55486976b003 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:43.838831 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-fe9d8d36-9dae-4333-b744-55486976b003 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:09:43.853619 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9d8d36-9dae-4333-b744-55486976b003 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:43.912121 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31b85cc-04ae-4373-be75-d1d3ab1a42b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:43.912745 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31b85cc-04ae-4373-be75-d1d3ab1a42b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:43.913018 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31b85cc-04ae-4373-be75-d1d3ab1a42b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:43.913281 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31b85cc-04ae-4373-be75-d1d3ab1a42b7 admin admin] PATH_INFO: `/v3/projects/c8a0b99a52d649959bbde0d378cb645a` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:43.914377 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31b85cc-04ae-4373-be75-d1d3ab1a42b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8a0b99a52d649959bbde0d378cb645a)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:43.916315 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31b85cc-04ae-4373-be75-d1d3ab1a42b7 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:43.962623 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-c31b85cc-04ae-4373-be75-d1d3ab1a42b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8a0b99a52d649959bbde0d378cb645a', 'request_id': 'req-c31b85cc-04ae-4373-be75-d1d3ab1a42b7'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:43.969961 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-c31b85cc-04ae-4373-be75-d1d3ab1a42b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8a0b99a52d649959bbde0d378cb645a', 'request_id': 'req-c31b85cc-04ae-4373-be75-d1d3ab1a42b7'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:43.976103 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-c31b85cc-04ae-4373-be75-d1d3ab1a42b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8a0b99a52d649959bbde0d378cb645a', 'request_id': 'req-c31b85cc-04ae-4373-be75-d1d3ab1a42b7'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:43.986475 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-c31b85cc-04ae-4373-be75-d1d3ab1a42b7 admin admin] The token cache is being invalidate because project c8a0b99a52d649959bbde0d378cb645a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 20:09:43.986756 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-c31b85cc-04ae-4373-be75-d1d3ab1a42b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:43.987370 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-c31b85cc-04ae-4373-be75-d1d3ab1a42b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:09:43.988875 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1483/2961] 10.222.0.61 () {58 vars in 1233 bytes} [Mon Aug 22 20:09:43 2022] DELETE /identity/v3/projects/c8a0b99a52d649959bbde0d378cb645a => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 20:09:43.993634 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04446f93-04ee-4a61-84a4-3f23df38cc1e None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:43.993921 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04446f93-04ee-4a61-84a4-3f23df38cc1e None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:43.994196 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04446f93-04ee-4a61-84a4-3f23df38cc1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:44.013319 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-04446f93-04ee-4a61-84a4-3f23df38cc1e None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 20:09:44.015033 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-04446f93-04ee-4a61-84a4-3f23df38cc1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:09:44.073077 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1479/2962] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 20:09:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 20:09:44.081205 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b562df53-538b-4987-9ba4-fea23140918b None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:44.081487 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b562df53-538b-4987-9ba4-fea23140918b None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:44.081759 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b562df53-538b-4987-9ba4-fea23140918b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:44.101231 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-b562df53-538b-4987-9ba4-fea23140918b None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 20:09:44.102420 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-b562df53-538b-4987-9ba4-fea23140918b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:09:44.110303 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1484/2963] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 20:09:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 20:09:44.114415 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04446f93-04ee-4a61-84a4-3f23df38cc1e None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:44.124122 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04446f93-04ee-4a61-84a4-3f23df38cc1e None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:44.128651 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be9c7c6-0e84-4dab-933e-27fb3a3e3224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:44.129238 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be9c7c6-0e84-4dab-933e-27fb3a3e3224 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:44.129539 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be9c7c6-0e84-4dab-933e-27fb3a3e3224 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:44.129808 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be9c7c6-0e84-4dab-933e-27fb3a3e3224 admin admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:44.130453 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be9c7c6-0e84-4dab-933e-27fb3a3e3224 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 20:09:44.130752 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be9c7c6-0e84-4dab-933e-27fb3a3e3224 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:44.132378 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be9c7c6-0e84-4dab-933e-27fb3a3e3224 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:44.144186 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1480/2964] 10.222.0.61 () {58 vars in 1153 bytes} [Mon Aug 22 20:09:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 20:09:44.148379 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b562df53-538b-4987-9ba4-fea23140918b None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:44.158209 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b562df53-538b-4987-9ba4-fea23140918b None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:44.162830 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96889e1-5b68-4fbb-bb27-e2da75e46470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:44.163418 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96889e1-5b68-4fbb-bb27-e2da75e46470 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:44.163723 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96889e1-5b68-4fbb-bb27-e2da75e46470 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:44.163995 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96889e1-5b68-4fbb-bb27-e2da75e46470 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:44.164618 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96889e1-5b68-4fbb-bb27-e2da75e46470 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:44.167115 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96889e1-5b68-4fbb-bb27-e2da75e46470 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:44.194542 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1485/2965] 10.222.0.61 () {60 vars in 1153 bytes} [Mon Aug 22 20:09:44 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 20:09:44.198963 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be9c7c6-0e84-4dab-933e-27fb3a3e3224 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:44.208936 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be9c7c6-0e84-4dab-933e-27fb3a3e3224 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:44.217659 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:09:44.266509 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3ee71f-e3d4-47ea-97a5-5cdb3c5ed29c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:44.267109 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3ee71f-e3d4-47ea-97a5-5cdb3c5ed29c admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:44.267388 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3ee71f-e3d4-47ea-97a5-5cdb3c5ed29c admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:44.267683 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3ee71f-e3d4-47ea-97a5-5cdb3c5ed29c admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:44.268302 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3ee71f-e3d4-47ea-97a5-5cdb3c5ed29c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:44.270214 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3ee71f-e3d4-47ea-97a5-5cdb3c5ed29c admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:44.290800 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1481/2966] 10.222.0.61 () {60 vars in 1144 bytes} [Mon Aug 22 20:09:44 2022] POST /identity/v3/users => generated 286 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 20:09:44.294844 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96889e1-5b68-4fbb-bb27-e2da75e46470 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:44.305549 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96889e1-5b68-4fbb-bb27-e2da75e46470 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:44.310023 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a77f6c-0fbc-498e-9b13-8bbd7d7805c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:44.310607 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a77f6c-0fbc-498e-9b13-8bbd7d7805c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:44.310882 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a77f6c-0fbc-498e-9b13-8bbd7d7805c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:44.311150 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a77f6c-0fbc-498e-9b13-8bbd7d7805c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:44.311828 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a77f6c-0fbc-498e-9b13-8bbd7d7805c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:44.313431 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a77f6c-0fbc-498e-9b13-8bbd7d7805c6 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:44.323578 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1486/2967] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 20:09:44 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 20:09:44.327462 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3ee71f-e3d4-47ea-97a5-5cdb3c5ed29c admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:44.337142 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3ee71f-e3d4-47ea-97a5-5cdb3c5ed29c admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:44.341680 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d074254-1092-46ce-8217-64ef25d5775f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:44.342258 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d074254-1092-46ce-8217-64ef25d5775f admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:44.343373 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d074254-1092-46ce-8217-64ef25d5775f admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:44.343664 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d074254-1092-46ce-8217-64ef25d5775f admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:44.344315 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d074254-1092-46ce-8217-64ef25d5775f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:44.345871 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d074254-1092-46ce-8217-64ef25d5775f admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:44.355458 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1482/2968] 10.222.0.61 () {58 vars in 1122 bytes} [Mon Aug 22 20:09:44 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 20:09:44.360152 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a77f6c-0fbc-498e-9b13-8bbd7d7805c6 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:44.370345 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a77f6c-0fbc-498e-9b13-8bbd7d7805c6 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:44.375009 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df93be7-b3bf-49ed-bd95-7c1b5c84c6de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:44.375685 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df93be7-b3bf-49ed-bd95-7c1b5c84c6de admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:44.375973 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df93be7-b3bf-49ed-bd95-7c1b5c84c6de admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:44.376250 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df93be7-b3bf-49ed-bd95-7c1b5c84c6de admin admin] PATH_INFO: `/v3/projects/c66fdae148a14b96873b833df4b98e08/users/f92c40c93f934b3f8acde6d1327f15ab/roles/74a40bc819cb42eea7e966a4b2a016f1` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:44.390621 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df93be7-b3bf-49ed-bd95-7c1b5c84c6de admin admin] RBAC: Authorizing `identity:create_grant(project_id=c66fdae148a14b96873b833df4b98e08, user_id=f92c40c93f934b3f8acde6d1327f15ab, role_id=74a40bc819cb42eea7e966a4b2a016f1)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:44.393282 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df93be7-b3bf-49ed-bd95-7c1b5c84c6de admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:44.406249 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1487/2969] 10.222.0.61 () {60 vars in 1483 bytes} [Mon Aug 22 20:09:44 2022] PUT /identity/v3/projects/c66fdae148a14b96873b833df4b98e08/users/f92c40c93f934b3f8acde6d1327f15ab/roles/74a40bc819cb42eea7e966a4b2a016f1 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 20:09:44.417346 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a5aa17-f5d8-4fc1-b789-b1ac6dc73f97 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:44.417842 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a5aa17-f5d8-4fc1-b789-b1ac6dc73f97 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:44.417944 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a5aa17-f5d8-4fc1-b789-b1ac6dc73f97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:44.445598 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-57a5aa17-f5d8-4fc1-b789-b1ac6dc73f97 None None] MFA Rules not processed for user `f92c40c93f934b3f8acde6d1327f15ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 20:09:44.447069 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-57a5aa17-f5d8-4fc1-b789-b1ac6dc73f97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:09:44.509429 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1483/2970] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 20:09:44 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 20:09:44.521307 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df93be7-b3bf-49ed-bd95-7c1b5c84c6de admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:09:44.522974 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-2df93be7-b3bf-49ed-bd95-7c1b5c84c6de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:09:44.543102 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=89,keystone:DELETE=8,keystone:INSERT=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:09:44.582551 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df93be7-b3bf-49ed-bd95-7c1b5c84c6de admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:09:44.586216 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0628f1-4911-4587-8f32-0520424b8925 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d6a79ed44e5396d13cd29ba160ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:09:44.586820 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0628f1-4911-4587-8f32-0520424b8925 service glance] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:09:44.587101 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0628f1-4911-4587-8f32-0520424b8925 service glance] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:09:44.587375 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0628f1-4911-4587-8f32-0520424b8925 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:09:44.596098 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0628f1-4911-4587-8f32-0520424b8925 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:09:44.597518 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0628f1-4911-4587-8f32-0520424b8925 service glance] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:09:44.606424 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1488/2971] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 20:09:44 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 20:09:54.506429 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=60,keystone:INSERT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:09:54.602237 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=18 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:10:20.682430 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a5aa17-f5d8-4fc1-b789-b1ac6dc73f97 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:10:20.746934 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a5aa17-f5d8-4fc1-b789-b1ac6dc73f97 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:10:20.751531 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7e774a-95cf-4f2e-9901-c7f4dd92f7ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:10:20.752760 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7e774a-95cf-4f2e-9901-c7f4dd92f7ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:10:20.753050 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7e774a-95cf-4f2e-9901-c7f4dd92f7ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:10:20.753311 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7e774a-95cf-4f2e-9901-c7f4dd92f7ac admin admin] PATH_INFO: `/v3/users/f92c40c93f934b3f8acde6d1327f15ab` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:10:20.754449 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7e774a-95cf-4f2e-9901-c7f4dd92f7ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=f92c40c93f934b3f8acde6d1327f15ab)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:10:20.756392 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7e774a-95cf-4f2e-9901-c7f4dd92f7ac admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:10:20.801921 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-eb7e774a-95cf-4f2e-9901-c7f4dd92f7ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f92c40c93f934b3f8acde6d1327f15ab'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:10:20.802564 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-eb7e774a-95cf-4f2e-9901-c7f4dd92f7ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f92c40c93f934b3f8acde6d1327f15ab'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:10:20.809373 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-eb7e774a-95cf-4f2e-9901-c7f4dd92f7ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f92c40c93f934b3f8acde6d1327f15ab'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:10:20.816602 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-eb7e774a-95cf-4f2e-9901-c7f4dd92f7ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f92c40c93f934b3f8acde6d1327f15ab'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:10:20.829911 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-eb7e774a-95cf-4f2e-9901-c7f4dd92f7ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f92c40c93f934b3f8acde6d1327f15ab'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:10:20.835777 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1484/2972] 10.222.0.61 () {58 vars in 1224 bytes} [Mon Aug 22 20:10:20 2022] DELETE /identity/v3/users/f92c40c93f934b3f8acde6d1327f15ab => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 20:10:20.840238 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cce4f98-89bc-4894-9f3d-96d0fcd41d6b None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:10:20.840724 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cce4f98-89bc-4894-9f3d-96d0fcd41d6b None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:10:20.840825 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cce4f98-89bc-4894-9f3d-96d0fcd41d6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:10:20.869294 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-1cce4f98-89bc-4894-9f3d-96d0fcd41d6b None None] MFA Rules not processed for user `d9c7158eb3ce41ab8bb7a5d05da4a2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 20:10:20.870578 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-1cce4f98-89bc-4894-9f3d-96d0fcd41d6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:10:20.933639 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1489/2973] 10.222.0.61 () {58 vars in 958 bytes} [Mon Aug 22 20:10:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 20:10:20.943201 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7e774a-95cf-4f2e-9901-c7f4dd92f7ac admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:10:20.944632 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-eb7e774a-95cf-4f2e-9901-c7f4dd92f7ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:10:21.008629 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7e774a-95cf-4f2e-9901-c7f4dd92f7ac admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:10:21.012339 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56528575-1f71-4928-aba1-7dfdff94e9c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aca06c966a42f29f5c5bc108529d41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:10:21.012938 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56528575-1f71-4928-aba1-7dfdff94e9c9 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:10:21.013219 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56528575-1f71-4928-aba1-7dfdff94e9c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:10:21.013493 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56528575-1f71-4928-aba1-7dfdff94e9c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:10:21.022384 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56528575-1f71-4928-aba1-7dfdff94e9c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:10:21.023822 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56528575-1f71-4928-aba1-7dfdff94e9c9 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:10:21.033891 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1485/2974] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 20:10:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 20:10:21.296541 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cce4f98-89bc-4894-9f3d-96d0fcd41d6b None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:10:21.297889 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-1cce4f98-89bc-4894-9f3d-96d0fcd41d6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:10:21.309911 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cce4f98-89bc-4894-9f3d-96d0fcd41d6b None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:10:21.314914 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401fac3b-5851-460a-b13d-929932325526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c7158eb3ce41ab8bb7a5d05da4a2f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bc536d83f9841a59997d0d04e993cd8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:10:21.315630 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401fac3b-5851-460a-b13d-929932325526 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:10:21.315993 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401fac3b-5851-460a-b13d-929932325526 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:10:21.316329 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401fac3b-5851-460a-b13d-929932325526 admin admin] PATH_INFO: `/v3/projects/c66fdae148a14b96873b833df4b98e08` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:10:21.317664 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401fac3b-5851-460a-b13d-929932325526 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c66fdae148a14b96873b833df4b98e08)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:10:21.319867 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401fac3b-5851-460a-b13d-929932325526 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:10:21.366821 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-401fac3b-5851-460a-b13d-929932325526 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c66fdae148a14b96873b833df4b98e08', 'request_id': 'req-401fac3b-5851-460a-b13d-929932325526'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:10:21.373293 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-401fac3b-5851-460a-b13d-929932325526 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c66fdae148a14b96873b833df4b98e08', 'request_id': 'req-401fac3b-5851-460a-b13d-929932325526'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:10:21.379874 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-401fac3b-5851-460a-b13d-929932325526 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c66fdae148a14b96873b833df4b98e08', 'request_id': 'req-401fac3b-5851-460a-b13d-929932325526'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:10:21.391385 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-401fac3b-5851-460a-b13d-929932325526 admin admin] The token cache is being invalidate because project c66fdae148a14b96873b833df4b98e08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 20:10:21.391761 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-401fac3b-5851-460a-b13d-929932325526 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:10:21.392866 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-401fac3b-5851-460a-b13d-929932325526 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 20:10:21.394321 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1490/2975] 10.222.0.61 () {58 vars in 1233 bytes} [Mon Aug 22 20:10:21 2022] DELETE /identity/v3/projects/c66fdae148a14b96873b833df4b98e08 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 20:10:31.029088 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=84,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:10:31.387859 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=58,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:13:12.457871 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56528575-1f71-4928-aba1-7dfdff94e9c9 service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:13:12.459299 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-56528575-1f71-4928-aba1-7dfdff94e9c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:13:12.527442 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56528575-1f71-4928-aba1-7dfdff94e9c9 service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:13:12.585147 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83b4611-8fd0-4396-a5b4-5ef0475945f0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eef0f73297e04d62bc22acb790fad6ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:13:12.585749 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83b4611-8fd0-4396-a5b4-5ef0475945f0 service placement] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:13:12.586028 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83b4611-8fd0-4396-a5b4-5ef0475945f0 service placement] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:13:12.586300 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83b4611-8fd0-4396-a5b4-5ef0475945f0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:13:12.588648 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-c83b4611-8fd0-4396-a5b4-5ef0475945f0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:13:12.599698 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83b4611-8fd0-4396-a5b4-5ef0475945f0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:13:12.601257 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83b4611-8fd0-4396-a5b4-5ef0475945f0 service placement] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:13:12.611191 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1486/2976] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 20:13:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 20:13:22.606930 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=58 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:15:15.464547 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ed5855-b000-4d3d-b6ff-b7a574ff0447 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:15:15.465161 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ed5855-b000-4d3d-b6ff-b7a574ff0447 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:15:15.465161 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ed5855-b000-4d3d-b6ff-b7a574ff0447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:15:15.503147 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-c9ed5855-b000-4d3d-b6ff-b7a574ff0447 None None] MFA Rules not processed for user `eef0f73297e04d62bc22acb790fad6ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 20:15:15.504406 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-c9ed5855-b000-4d3d-b6ff-b7a574ff0447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:15:15.512964 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1491/2977] 10.222.0.16 () {58 vars in 1015 bytes} [Mon Aug 22 20:15:15 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 20:15:15.525642 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83b4611-8fd0-4396-a5b4-5ef0475945f0 service placement] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 20:15:15.527001 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-c83b4611-8fd0-4396-a5b4-5ef0475945f0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 20:15:15.539789 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83b4611-8fd0-4396-a5b4-5ef0475945f0 service placement] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 20:15:15.545321 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39233075-ac3f-422f-89db-6608d7ff2bc9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eef0f73297e04d62bc22acb790fad6ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e07dc5ebdc504c6da706a2d21c0a3970', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 20:15:15.545909 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39233075-ac3f-422f-89db-6608d7ff2bc9 service placement] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 20:15:15.546186 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39233075-ac3f-422f-89db-6608d7ff2bc9 service placement] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 20:15:15.546456 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39233075-ac3f-422f-89db-6608d7ff2bc9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 20:15:15.554723 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39233075-ac3f-422f-89db-6608d7ff2bc9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 20:15:15.556919 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39233075-ac3f-422f-89db-6608d7ff2bc9 service placement] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 20:15:15.566739 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1487/2978] 10.222.0.61 () {60 vars in 1365 bytes} [Mon Aug 22 20:15:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 20:15:25.500771 dv-cinder-835384-22-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=9 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 20:15:25.562412 dv-cinder-835384-22-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=6 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}